honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[DOCS] Updates dataframe transform terminology (#46642)

This commit is contained in:
Lisa Cawley 2019-09-16 08:28:19 -07:00 committed by lcawl
parent 92acc732de
commit dddc9b3d73
33 changed files with 357 additions and 357 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/java-rest/high-level/dataframe/delete_data_frame.asciidoc
View File

@ -5,12 +5,12 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Delete {dataframe-transform} API
=== Delete {transform} API
Deletes an existing {dataframe-transform}.
Deletes an existing {transform}.
[id="{upid}-{api}-request"]
==== Delete {dataframe-transform} request
==== Delete {transform} request
A +{request}+ object requires a non-null `id`.
@ -18,14 +18,14 @@ A +{request}+ object requires a non-null `id`.
---------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request]
---------------------------------------------------
<1> Constructing a new request referencing an existing {dataframe-transform}
<2> Sets the optional argument `force`. When `true`, the {dataframe-transform}
<1> Constructing a new request referencing an existing {transform}
<2> Sets the optional argument `force`. When `true`, the {transform}
is deleted regardless of its current state. The default value is `false`,
meaning that only `stopped` {dataframe-transforms} can be deleted.
meaning that only `stopped` {transforms} can be deleted.
include::../execution.asciidoc[]
[id="{upid}-{api}-response"]
==== Response
The returned +{response}+ object acknowledges the Data Frame Transform deletion.
The returned +{response}+ object acknowledges the {transform} deletion.

20
docs/java-rest/high-level/dataframe/get_data_frame.asciidoc
View File

@ -5,22 +5,22 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Get {dataframe-transform} API
=== Get {transform} API
Retrieves configuration information about one or more {dataframe-transforms}.
Retrieves configuration information about one or more {transforms}.
The API accepts a +{request}+ object and returns a +{response}+.
[id="{upid}-{api}-request"]
==== Get {dataframe-transform} request
==== Get {transform} request
A +{request}+ requires either a {dataframe-transform} ID, a comma separated list
of ids or the special wildcard `_all` to get all {dataframe-transforms}.
A +{request}+ requires either a {transform} ID, a comma separated list
of ids or the special wildcard `_all` to get all {transforms}.
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request]
--------------------------------------------------
<1> Constructing a new GET request referencing an existing {dataframe-transform}
<1> Constructing a new GET request referencing an existing {transform}
==== Optional arguments
@ -31,9 +31,9 @@ The following arguments are optional.
include-tagged::{doc-tests-file}[{api}-request-options]
--------------------------------------------------
<1> The page parameters `from` and `size`. `from` specifies the number of
{dataframe-transforms} to skip. `size` specifies the maximum number of
{dataframe-transforms} to get. Defaults to `0` and `100` respectively.
<2> Whether to ignore if a wildcard expression matches no transforms.
{transforms} to skip. `size` specifies the maximum number of
{transforms} to get. Defaults to `0` and `100` respectively.
<2> Whether to ignore if a wildcard expression matches no {transforms}.
include::../execution.asciidoc[]
@ -41,7 +41,7 @@ include::../execution.asciidoc[]
[id="{upid}-{api}-response"]
==== Response
The returned +{response}+ contains the requested {dataframe-transforms}.
The returned +{response}+ contains the requested {transforms}.
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------

26
docs/java-rest/high-level/dataframe/get_data_frame_stats.asciidoc
View File

@ -5,22 +5,22 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Get {dataframe-transform} stats API
=== Get {transform} stats API
Retrieves the operational statistics of one or more {dataframe-transforms}.
Retrieves the operational statistics of one or more {transforms}.
The API accepts a +{request}+ object and returns a +{response}+.
[id="{upid}-{api}-request"]
==== Get {dataframe-transform} stats request
==== Get {transform} stats request
A +{request}+ requires a data frame transform id or the special wildcard `_all`
to get the statistics for all {dataframe-transforms}.
A +{request}+ requires a {transform} id or the special wildcard `_all`
to get the statistics for all {transforms}.
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request]
--------------------------------------------------
<1> Constructing a new GET Stats request referencing an existing {dataframe-transform}
<1> Constructing a new GET Stats request referencing an existing {transform}
==== Optional arguments
@ -31,10 +31,10 @@ The following arguments are optional.
include-tagged::{doc-tests-file}[{api}-request-options]
--------------------------------------------------
<1> The page parameters `from` and `size`. `from` specifies the number of
{dataframe-transform} stats to skip.
`size` specifies the maximum number of {dataframe-transform} stats to get.
{transform} stats to skip.
`size` specifies the maximum number of {transform} stats to get.
Defaults to `0` and `100` respectively.
<2> Whether to ignore if a wildcard expression matches no transforms.
<2> Whether to ignore if a wildcard expression matches no {transforms}.
include::../execution.asciidoc[]
@ -42,15 +42,15 @@ include::../execution.asciidoc[]
[id="{upid}-{api}-response"]
==== Response
The returned +{response}+ contains the requested {dataframe-transform} statistics.
The returned +{response}+ contains the requested {transform} statistics.
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-response]
--------------------------------------------------
<1> The response contains a list of `DataFrameTransformStats` objects
<2> The running state of the transform, for example `started`, `indexing`, etc.
<3> The overall transform statistics recording the number of documents indexed etc.
<4> The progress of the current run in the transform. Supplies the number of docs left until the next checkpoint
<2> The running state of the {transform}, for example `started`, `indexing`, etc.
<3> The overall {transform} statistics recording the number of documents indexed etc.
<4> The progress of the current run in the {transform}. Supplies the number of docs left until the next checkpoint
and the total number of docs expected.
<5> The assigned node information if the task is currently assigned to a node and running.

10
docs/java-rest/high-level/dataframe/preview_data_frame.asciidoc
View File

@ -5,16 +5,16 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Preview {dataframe-transform} API
=== Preview {transform} API
Previews the results of a {dataframe-transform}.
Previews the results of a {transform}.
The API accepts a +{request}+ object as a request and returns a +{response}+.
[id="{upid}-{api}-request"]
==== Preview {dataframe-transform} request
==== Preview {transform} request
A +{request}+ takes a single argument: a valid {dataframe-transform} config.
A +{request}+ takes a single argument: a valid {transform} config.
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
@ -22,7 +22,7 @@ include-tagged::{doc-tests-file}[{api}-request]
--------------------------------------------------
<1> The source config from which the data should be gathered
<2> The pivot config used to transform the data
<3> The configuration of the {dataframe-transform} to preview
<3> The configuration of the {transform} to preview
include::../execution.asciidoc[]

24
docs/java-rest/high-level/dataframe/put_data_frame.asciidoc
View File

@ -5,14 +5,14 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Put {dataframe-transform} API
=== Put {transform} API
Creates a new {dataframe-transform}.
Creates a new {transform}.
The API accepts a +{request}+ object as a request and returns a +{response}+.
[id="{upid}-{api}-request"]
==== Put {dataframe-transform} request
==== Put {transform} request
A +{request}+ requires the following argument:
@ -20,28 +20,28 @@ A +{request}+ requires the following argument:
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request]
--------------------------------------------------
<1> The configuration of the {dataframe-transform} to create
<1> The configuration of the {transform} to create
<2> Whether or not to wait to run deferrable validations until `_start` is called.
This option should be used with care as the created {dataframe-transform} will run
This option should be used with care as the created {transform} will run
with the privileges of the user creating it. Meaning, if they do not have privileges,
such an error will not be visible until `_start` is called.
[id="{upid}-{api}-config"]
==== {dataframe-transform-cap} configuration
==== {transform-cap} configuration
The `DataFrameTransformConfig` object contains all the details about the
{dataframe-transform} configuration and contains the following arguments:
{transform} configuration and contains the following arguments:
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-config]
--------------------------------------------------
<1> The {dataframe-transform} ID
<1> The {transform} ID
<2> The source indices and query from which to gather data
<3> The destination index and optional pipeline
<4> How often to check for updates to the source indices
<5> The PivotConfig
<6> Optional free text description of the transform
<6> Optional free text description of the {transform}
[id="{upid}-{api}-query-config"]
@ -84,8 +84,8 @@ include-tagged::{doc-tests-file}[{api}-pivot-config]
--------------------------------------------------
<1> The `GroupConfig` to use in the pivot
<2> The aggregations to use
<3> The maximum paging size for the transform when pulling data
from the source. The size dynamically adjusts as the transform
<3> The maximum paging size for the {transform} when pulling data
from the source. The size dynamically adjusts as the {transform}
is running to recover from and prevent OOM issues.
===== GroupConfig
@ -121,4 +121,4 @@ include::../execution.asciidoc[]
==== Response
The returned +{response}+ acknowledges the successful creation of
the new {dataframe-transform} or an error if the configuration is invalid.
the new {transform} or an error if the configuration is invalid.

12
docs/java-rest/high-level/dataframe/start_data_frame.asciidoc
View File

@ -5,13 +5,13 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Start {dataframe-transform} API
=== Start {transform} API
Starts a {dataframe-transform}.
Starts a {transform}.
It accepts a +{request}+ object and responds with a +{response}+ object.
[id="{upid}-{api}-request"]
==== Start {dataframe-transform} request
==== Start {transform} request
A +{request}+ object requires a non-null `id`.
@ -20,7 +20,7 @@ A +{request}+ object requires a non-null `id`.
include-tagged::{doc-tests-file}[{api}-request]
---------------------------------------------------
<1> Constructing a new start request referencing an existing
{dataframe-transform}
{transform}
==== Optional arguments
@ -30,11 +30,11 @@ The following arguments are optional.
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request-options]
--------------------------------------------------
<1> Controls the amount of time to wait until the {dataframe-transform} starts.
<1> Controls the amount of time to wait until the {transform} starts.
include::../execution.asciidoc[]
==== Response
The returned +{response}+ object acknowledges the {dataframe-transform} has
The returned +{response}+ object acknowledges the {transform} has
started.

14
docs/java-rest/high-level/dataframe/stop_data_frame.asciidoc
View File

@ -5,13 +5,13 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Stop {dataframe-transform} API
=== Stop {transform} API
Stops a started {dataframe-transform}.
Stops a started {transform}.
It accepts a +{request}+ object and responds with a +{response}+ object.
[id="{upid}-{api}-request"]
==== Stop {dataframe-transform} request
==== Stop {transform} request
A +{request}+ object requires a non-null `id`. `id` can be a comma separated
list of IDs or a single ID. Wildcards, `*` and `_all` are also accepted.
@ -21,7 +21,7 @@ list of IDs or a single ID. Wildcards, `*` and `_all` are also accepted.
---------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request]
---------------------------------------------------
<1> Constructing a new stop request referencing an existing {dataframe-transform}
<1> Constructing a new stop request referencing an existing {transform}
==== Optional arguments
@ -32,11 +32,11 @@ The following arguments are optional.
include-tagged::{doc-tests-file}[{api}-request-options]
--------------------------------------------------
<1> If true wait for the data frame task to stop before responding
<2> Controls the amount of time to wait until the {dataframe-transform} stops.
<3> Whether to ignore if a wildcard expression matches no transforms.
<2> Controls the amount of time to wait until the {transform} stops.
<3> Whether to ignore if a wildcard expression matches no {transforms}.
include::../execution.asciidoc[]
==== Response
The returned +{response}+ object acknowledges the {dataframe-transform} has stopped.
The returned +{response}+ object acknowledges the {transform} has stopped.

20
docs/java-rest/high-level/dataframe/update_data_frame.asciidoc
View File

@ -5,14 +5,14 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Update {dataframe-transform} API
=== Update {transform} API
Updates an existing {dataframe-transform}.
Updates an existing {transform}.
The API accepts a +{request}+ object as a request and returns a +{response}+.
[id="{upid}-{api}-request"]
==== Update {dataframe-transform} request
==== Update {transform} request
A +{request}+ requires the following argument:
@ -20,18 +20,18 @@ A +{request}+ requires the following argument:
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request]
--------------------------------------------------
<1> The update configuration with which to update the {dataframe-transform}.
<1> The update configuration with which to update the {transform}.
<2> The ID of the configuration to update.
<3> Whether or not to wait to run deferrable validations until `_start` is called.
This option should be used with care as the created {dataframe-transform} will run
This option should be used with care as the created {transform} will run
with the privileges of the user creating it. Meaning, if they do not have privileges,
such an error will not be visible until `_start` is called.
[id="{upid}-{api}-config"]
==== {dataframe-transform-cap} update configuration
==== {transform-cap} update configuration
The `DataFrameTransformConfigUpdate` object contains all the details about updated
{dataframe-transform} configuration and contains the following arguments:
{transform} configuration and contains the following arguments:
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
@ -40,13 +40,13 @@ include-tagged::{doc-tests-file}[{api}-config]
<1> The source indices and query from which to gather data.
<2> The destination index and optional pipeline.
<3> How often to check for updates to the source indices.
<4> How to keep the {dataframe-transform} in sync with incoming data.
<5> Optional free text description of the transform.
<4> How to keep the {transform} in sync with incoming data.
<5> Optional free text description of the {transform}.
include::../execution.asciidoc[]
[id="{upid}-{api}-response"]
==== Response
The returned +{response}+ contains the updated {dataframe-transform} configuration
The returned +{response}+ contains the updated {transform} configuration
or an error if the update failed or is invalid.

16
docs/java-rest/high-level/ml/put-data-frame-analytics.asciidoc
View File

@ -5,13 +5,13 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Put Data Frame Analytics API
=== Put {dfanalytics} API
The Put Data Frame Analytics API is used to create a new {dataframe-analytics-config}.
The Put {dfanalytics} API is used to create a new {dataframe-analytics-config}.
The API accepts a +{request}+ object as a request and returns a +{response}+.
[id="{upid}-{api}-request"]
==== Put Data Frame Analytics Request
==== Put {dfanalytics} request
A +{request}+ requires the following argument:
@ -19,12 +19,12 @@ A +{request}+ requires the following argument:
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request]
--------------------------------------------------
<1> The configuration of the {dataframe-job} to create
<1> The configuration of the {dfanalytics-job} to create
[id="{upid}-{api}-config"]
==== Data Frame Analytics Configuration
==== {dfanalytics-cap} configuration
The `DataFrameAnalyticsConfig` object contains all the details about the {dataframe-job}
The `DataFrameAnalyticsConfig` object contains all the details about the {dfanalytics-job}
configuration and contains the following arguments:
["source","java",subs="attributes,callouts,macros"]
@ -64,7 +64,7 @@ include-tagged::{doc-tests-file}[{api}-query-config]
==== DestinationConfig
The index to which data should be written by the {dataframe-job}.
The index to which data should be written by the {dfanalytics-job}.
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
@ -78,7 +78,7 @@ include-tagged::{doc-tests-file}[{api}-dest-config]
The analysis to be performed.
Currently, the supported analyses include : +OutlierDetection+, +Regression+.
===== Outlier Detection
===== Outlier detection
+OutlierDetection+ analysis can be created in one of two ways:

8
docs/java-rest/high-level/ml/start-data-frame-analytics.asciidoc
View File

@ -5,13 +5,13 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Start Data Frame Analytics API
=== Start {dfanalytics} API
The Start Data Frame Analytics API is used to start an existing {dataframe-analytics-config}.
The start {dfanalytics} API is used to start an existing {dataframe-analytics-config}.
It accepts a +{request}+ object and responds with a +{response}+ object.
[id="{upid}-{api}-request"]
==== Start Data Frame Analytics Request
==== Start {dfanalytics} Request
A +{request}+ object requires a {dataframe-analytics-config} id.
@ -26,4 +26,4 @@ include::../execution.asciidoc[]
[id="{upid}-{api}-response"]
==== Response
The returned +{response}+ object acknowledges the {dataframe-job} has started.
The returned +{response}+ object acknowledges the {dfanalytics-job} has started.

8
docs/java-rest/high-level/ml/stop-data-frame-analytics.asciidoc
View File

@ -5,13 +5,13 @@
--
[role="xpack"]
[id="{upid}-{api}"]
=== Stop Data Frame Analytics API
=== Stop {dfanalytics} API
The Stop Data Frame Analytics API is used to stop a running {dataframe-analytics-config}.
The stop {dfanalytics} API is used to stop a running {dataframe-analytics-config}.
It accepts a +{request}+ object and responds with a +{response}+ object.
[id="{upid}-{api}-request"]
==== Stop Data Frame Analytics Request
==== Stop {dfanalytics} Request
A +{request}+ object requires a {dataframe-analytics-config} id.
@ -27,4 +27,4 @@ include::../execution.asciidoc[]
[id="{upid}-{api}-response"]
==== Response
The returned +{response}+ object acknowledges the {dataframe-job} has stopped.
The returned +{response}+ object acknowledges the {dfanalytics-job} has stopped.

4
docs/java-rest/high-level/supported-apis.asciidoc
View File

@ -577,12 +577,12 @@ include::ilm/remove_lifecycle_policy_from_index.asciidoc[]
[role="xpack"]
[[_data_frame_transform_apis]]
== {dataframe-transform-cap} APIs
== {transform-cap} APIs
:upid: {mainid}-dataframe
:doc-tests-file: {doc-tests}/DataFrameTransformDocumentationIT.java
The Java High Level REST Client supports the following {dataframe-transform}
The Java High Level REST Client supports the following {transform}
APIs:
* <<{upid}-get-data-frame-transform>>

20
docs/reference/data-frames/apis/delete-transform.asciidoc
View File

@ -1,26 +1,26 @@
[role="xpack"]
[testenv="basic"]
[[delete-data-frame-transform]]
=== Delete {dataframe-transforms} API
=== Delete {transforms} API
[subs="attributes"]
++++
<titleabbrev>Delete {dataframe-transforms}</titleabbrev>
<titleabbrev>Delete {transforms}</titleabbrev>
++++
Deletes an existing {dataframe-transform}.
Deletes an existing {transform}.
beta[]
[[delete-data-frame-transform-request]]
==== {api-request-title}
`DELETE _data_frame/transforms/<data_frame_transform_id>`
`DELETE _data_frame/transforms/<transform_id>`
[[delete-data-frame-transform-prereqs]]
==== {api-prereq-title}
* Before you can delete the {dataframe-transform}, you must stop it.
* Before you can delete the {transform}, you must stop it.
* If the {es} {security-features} are enabled, you must have
`manage_data_frame_transforms` cluster privileges to use this API. The built-in
`data_frame_transforms_admin` role has these privileges. For more information,
@ -31,15 +31,15 @@ see {stack-ov}/security-privileges.html[Security privileges] and
[[delete-data-frame-transform-path-parms]]
==== {api-path-parms-title}
`<data_frame_transform_id>`::
(Required, string) Identifier for the {dataframe-transform}.
`<transform_id>`::
(Required, string) Identifier for the {transform}.
[[delete-data-frame-transform-query-parms]]
==== {api-query-parms-title}
`force`::
(Optional, boolean) When `true`, the {dataframe-transform} is deleted regardless of its
current state. The default value is `false`, meaning that the {dataframe-transform} must be
(Optional, boolean) When `true`, the {transform} is deleted regardless of its
current state. The default value is `false`, meaning that the {transform} must be
`stopped` before it can be deleted.
[[delete-data-frame-transform-examples]]
@ -51,7 +51,7 @@ DELETE _data_frame/transforms/ecommerce_transform
--------------------------------------------------
// TEST[skip:setup kibana sample data]
When the {dataframe-transform} is deleted, you receive the following results:
When the {transform} is deleted, you receive the following results:
[source,console-result]
----

38
docs/reference/data-frames/apis/get-transform-stats.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[get-data-frame-transform-stats]]
=== Get {dataframe-transform} statistics API
=== Get {transform} statistics API
[subs="attributes"]
++++
<titleabbrev>Get {dataframe-transform} statistics</titleabbrev>
<titleabbrev>Get {transform} statistics</titleabbrev>
++++
Retrieves usage information for {dataframe-transforms}.
Retrieves usage information for {transforms}.
beta[]
@ -16,9 +16,9 @@ beta[]
[[get-data-frame-transform-stats-request]]
==== {api-request-title}
`GET _data_frame/transforms/<data_frame_transform_id>/_stats`
`GET _data_frame/transforms/<transform_id>/_stats`
`GET _data_frame/transforms/<data_frame_transform_id>,<data_frame_transform_id>/_stats` +
`GET _data_frame/transforms/<transform_id>,<transform_id>/_stats` +
`GET _data_frame/transforms/_stats` +
@ -40,21 +40,21 @@ see {stack-ov}/security-privileges.html[Security privileges] and
[[get-data-frame-transform-stats-desc]]
==== {api-description-title}
You can get statistics for multiple {dataframe-transforms} in a single API
You can get statistics for multiple {transforms} in a single API
request by using a comma-separated list of identifiers or a wildcard expression.
You can get statistics for all {dataframe-transforms} by using `_all`, by
specifying `*` as the `<data_frame_transform_id>`, or by omitting the
`<data_frame_transform_id>`.
You can get statistics for all {transforms} by using `_all`, by
specifying `*` as the `<transform_id>`, or by omitting the
`<transform_id>`.
[[get-data-frame-transform-stats-path-parms]]
==== {api-path-parms-title}
`<data_frame_transform_id>`::
(Optional, string) Identifier for the {dataframe-transform}. It can be a
{dataframe-transform} identifier or a wildcard expression. If you do not
`<transform_id>`::
(Optional, string) Identifier for the {transform}. It can be a
{transform} identifier or a wildcard expression. If you do not
specify one of these options, the API returns information for all
{dataframe-transforms}.
{transforms}.
[[get-data-frame-transform-stats-query-parms]]
@ -64,7 +64,7 @@ specifying `*` as the `<data_frame_transform_id>`, or by omitting the
(Optional, boolean) Specifies what to do when the request:
+
--
* Contains wildcard expressions and there are no {dataframe-transforms} that match.
* Contains wildcard expressions and there are no {transforms} that match.
* Contains the `_all` string or no identifiers and there are no matches.
* Contains wildcard expressions and there are only partial matches.
@ -75,17 +75,17 @@ are no matches or only partial matches.
--
`from`::
(Optional, integer) Skips the specified number of {dataframe-transforms}. The
(Optional, integer) Skips the specified number of {transforms}. The
default value is `0`.
`size`::
(Optional, integer) Specifies the maximum number of {dataframe-transforms} to obtain. The default value is `100`.
(Optional, integer) Specifies the maximum number of {transforms} to obtain. The default value is `100`.
[[get-data-frame-transform-stats-response]]
==== {api-response-body-title}
`transforms`::
(array) An array of statistics objects for {dataframe-transforms}, which are
(array) An array of statistics objects for {transforms}, which are
sorted by the `id` value in ascending order.
[[get-data-frame-transform-stats-response-codes]]
@ -98,7 +98,7 @@ are no matches or only partial matches.
[[get-data-frame-transform-stats-example]]
==== Examples
The following example skips for the first five {dataframe-transforms} and
The following example skips for the first five {transforms} and
gets usage information for a maximum of ten results:
[source,console]
@ -108,7 +108,7 @@ GET _data_frame/transforms/_stats?from=5&size=10
// TEST[skip:todo]
The following example gets usage information for the `ecommerce_transform`
{dataframe-transform}:
{transform}:
[source,console]
--------------------------------------------------

28
docs/reference/data-frames/apis/get-transform.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[get-data-frame-transform]]
=== Get {dataframe-transforms} API
=== Get {transforms} API
[subs="attributes"]
++++
<titleabbrev>Get {dataframe-transforms}</titleabbrev>
<titleabbrev>Get {transforms}</titleabbrev>
++++
Retrieves configuration information for {dataframe-transforms}.
Retrieves configuration information for {transforms}.
beta[]
@ -37,9 +37,9 @@ see {stack-ov}/security-privileges.html[Security privileges] and
[[get-data-frame-transform-desc]]
==== {api-description-title}
You can get information for multiple {dataframe-transforms} in a single API
You can get information for multiple {transforms} in a single API
request by using a comma-separated list of identifiers or a wildcard expression.
You can get information for all {dataframe-transforms} by using `_all`, by
You can get information for all {transforms} by using `_all`, by
specifying `*` as the `<data_frame_transform_id>`, or by omitting the
`<data_frame_transform_id>`.
@ -47,10 +47,10 @@ specifying `*` as the `<data_frame_transform_id>`, or by omitting the
==== {api-path-parms-title}
`<data_frame_transform_id>`::
(Optional, string) Identifier for the {dataframe-transform}. It can be a
{dataframe-transform} identifier or a wildcard expression. If you do not
(Optional, string) Identifier for the {transform}. It can be a
{transform} identifier or a wildcard expression. If you do not
specify one of these options, the API returns information for all
{dataframe-transforms}.
{transforms}.
[[get-data-frame-transform-query-parms]]
==== {api-query-parms-title}
@ -59,7 +59,7 @@ specifying `*` as the `<data_frame_transform_id>`, or by omitting the
(Optional, boolean) Specifies what to do when the request:
+
--
* Contains wildcard expressions and there are no {dataframe-transforms} that match.
* Contains wildcard expressions and there are no {transforms} that match.
* Contains the `_all` string or no identifiers and there are no matches.
* Contains wildcard expressions and there are only partial matches.
@ -70,17 +70,17 @@ are no matches or only partial matches.
--
`from`::
(Optional, integer) Skips the specified number of {dataframe-transforms}. The
(Optional, integer) Skips the specified number of {transforms}. The
default value is `0`.
`size`::
(Optional, integer) Specifies the maximum number of {dataframe-transforms} to obtain. The default value is `100`.
(Optional, integer) Specifies the maximum number of {transforms} to obtain. The default value is `100`.
[[get-data-frame-transform-response]]
==== {api-response-body-title}
`transforms`::
(array) An array of transform resources, which are sorted by the `id` value in
(array) An array of {transform} resources, which are sorted by the `id` value in
ascending order. See <<data-frame-transform-resource>>.
[[get-data-frame-transform-response-codes]]
@ -93,7 +93,7 @@ are no matches or only partial matches.
[[get-data-frame-transform-example]]
==== {api-examples-title}
The following example retrieves information about a maximum of ten transforms:
The following example retrieves information about a maximum of ten {transforms}:
[source,console]
--------------------------------------------------
@ -102,7 +102,7 @@ GET _data_frame/transforms?size=10
// TEST[skip:setup kibana sample data]
The following example gets configuration information for the
`ecommerce_transform` {dataframe-transform}:
`ecommerce_transform` {transform}:
[source,console]
--------------------------------------------------

4
docs/reference/data-frames/apis/index.asciidoc
View File

@ -1,9 +1,9 @@
[role="xpack"]
[testenv="basic"]
[[data-frame-apis]]
== {dataframe-transform-cap} APIs
== {transform-cap} APIs
See also {stack-ov}/ml-dataframes.html[{dataframe-transforms-cap}].
See also {stack-ov}/ml-dataframes.html[{transforms-cap}].
* <<put-data-frame-transform>>
* <<update-data-frame-transform>>

14
docs/reference/data-frames/apis/preview-transform.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[preview-data-frame-transform]]
=== Preview {dataframe-transforms} API
=== Preview {transforms} API
[subs="attributes"]
++++
<titleabbrev>Preview {dataframe-transforms}</titleabbrev>
<titleabbrev>Preview {transforms}</titleabbrev>
++++
Previews a {dataframe-transform}.
Previews a {transform}.
beta[]
@ -24,7 +24,7 @@ beta[]
`manage_data_frame_transforms` cluster privileges to use this API. The built-in
`data_frame_transforms_admin` role has these privileges. You must also have
`read` and `view_index_metadata` privileges on the source index for the
{dataframe-transform}. For more information, see
{transform}. For more information, see
{stack-ov}/security-privileges.html[Security privileges] and
{stack-ov}/built-in-roles.html[Built-in roles].
@ -32,7 +32,7 @@ beta[]
==== {api-description-title}
This API generates a preview of the results that you will get when you run the
<<put-data-frame-transform,create {dataframe-transforms} API>> with the same
<<put-data-frame-transform,create {transforms} API>> with the same
configuration. It returns a maximum of 100 results. The calculations are based
on all the current data in the source index.
@ -45,7 +45,7 @@ on all the current data in the source index.
`index`:::
(Required, string or array) The _source indices_ for the
{dataframe-transform}. It can be a single index, an index pattern (for
{transform}. It can be a single index, an index pattern (for
example, `"myindex*"`), or an array of indices (for example,
`["index1", "index2"]`).
@ -63,7 +63,7 @@ on all the current data in the source index.
`preview`::
(array) An array of documents. In particular, they are the JSON
representation of the documents that would be created in the destination index
by the {dataframe-transform}.
by the {transform}.
==== {api-examples-title}

40
docs/reference/data-frames/apis/put-transform.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[put-data-frame-transform]]
=== Create {dataframe-transforms} API
=== Create {transforms} API
[subs="attributes"]
++++
<titleabbrev>Create {dataframe-transforms}</titleabbrev>
<titleabbrev>Create {transforms}</titleabbrev>
++++
Instantiates a {dataframe-transform}.
Instantiates a {transform}.
beta[]
@ -31,30 +31,30 @@ information, see {stack-ov}/security-privileges.html[Security privileges] and
[[put-data-frame-transform-desc]]
==== {api-description-title}
This API defines a {dataframe-transform}, which copies data from source indices,
This API defines a {transform}, which copies data from source indices,
transforms it, and persists it into an entity-centric destination index. The
entities are defined by the set of `group_by` fields in the `pivot` object. You
can also think of the destination index as a two-dimensional tabular data
structure (known as a {dataframe}). The ID for each document in the
{dataframe} is generated from a hash of the entity, so there is a unique row
per entity. For more information, see
{stack-ov}/ml-dataframes.html[{dataframe-transforms-cap}].
{stack-ov}/ml-dataframes.html[{transforms-cap}].
When the {dataframe-transform} is created, a series of validations occur to
When the {transform} is created, a series of validations occur to
ensure its success. For example, there is a check for the existence of the
source indices and a check that the destination index is not part of the source
index pattern. You can use the `defer_validation` parameter to skip these
checks.
Deferred validations are always run when the {dataframe-transform} is started,
Deferred validations are always run when the {transform} is started,
with the exception of privilege checks. When {es} {security-features} are
enabled, the {dataframe-transform} remembers which roles the user that created
enabled, the {transform} remembers which roles the user that created
it had at the time of creation and uses those same roles. If those roles do not
have the required privileges on the source and destination indices, the
{dataframe-transform} fails when it attempts unauthorized operations.
{transform} fails when it attempts unauthorized operations.
IMPORTANT: You must use {kib} or this API to create a {dataframe-transform}.
Do not put a {dataframe-transform} directly into any
IMPORTANT: You must use {kib} or this API to create a {transform}.
Do not put a {transform} directly into any
`.data-frame-internal*` indices using the Elasticsearch index API.
If {es} {security-features} are enabled, do not give users any
privileges on `.data-frame-internal*` indices.
@ -63,7 +63,7 @@ IMPORTANT: You must use {kib} or this API to create a {dataframe-transform}.
==== {api-path-parms-title}
`<data_frame_transform_id>`::
(Required, string) Identifier for the {dataframe-transform}. This identifier
(Required, string) Identifier for the {transform}. This identifier
can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and
underscores. It must start and end with alphanumeric characters.
@ -73,28 +73,28 @@ IMPORTANT: You must use {kib} or this API to create a {dataframe-transform}.
`defer_validation`::
(Optional, boolean) When `true`, deferrable validations are not run. This
behavior may be desired if the source index does not exist until after the
{dataframe-transform} is created.
{transform} is created.
[[put-data-frame-transform-request-body]]
==== {api-request-body-title}
`description`::
(Optional, string) Free text description of the {dataframe-transform}.
(Optional, string) Free text description of the {transform}.
`dest`::
(Required, object) Required. The destination configuration, which has the
following properties:
`index`:::
(Required, string) The _destination index_ for the {dataframe-transform}.
(Required, string) The _destination index_ for the {transform}.
`pipeline`:::
(Optional, string) The unique identifier for a <<pipeline,pipeline>>.
`frequency`::
(Optional, <<time-units, time units>>) The interval between checks for changes in the source
indices when the {dataframe-transform} is running continuously. Also determines
the retry interval in the event of transient failures while the {dataframe-transform} is
indices when the {transform} is running continuously. Also determines
the retry interval in the event of transient failures while the {transform} is
searching or indexing. The minimum value is `1s` and the maximum is `1h`. The
default value is `1m`.
@ -108,7 +108,7 @@ IMPORTANT: You must use {kib} or this API to create a {dataframe-transform}.
`index`:::
(Required, string or array) The _source indices_ for the
{dataframe-transform}. It can be a single index, an index pattern (for
{transform}. It can be a single index, an index pattern (for
example, `"myindex*"`), or an array of indices (for example,
`["index1", "index2"]`).
@ -119,7 +119,7 @@ IMPORTANT: You must use {kib} or this API to create a {dataframe-transform}.
`sync`::
(Optional, object) Defines the properties required to run continuously.
`time`:::
(Required, object) Specifies that the {dataframe-transform} uses a time
(Required, object) Specifies that the {transform} uses a time
field to synchronize the source and destination indices.
`field`::::
(Required, string) The date field that is used to identify new documents
@ -185,7 +185,7 @@ PUT _data_frame/transforms/ecommerce_transform
--------------------------------------------------
// TEST[setup:kibana_sample_data_ecommerce]
When the transform is created, you receive the following results:
When the {transform} is created, you receive the following results:
[source,console-result]
----

26
docs/reference/data-frames/apis/start-transform.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[start-data-frame-transform]]
=== Start {dataframe-transforms} API
=== Start {transforms} API
[subs="attributes"]
++++
<titleabbrev>Start {dataframe-transforms}</titleabbrev>
<titleabbrev>Start {transforms}</titleabbrev>
++++
Starts one or more {dataframe-transforms}.
Starts one or more {transforms}.
beta[]
@ -23,38 +23,38 @@ beta[]
* If the {es} {security-features} are enabled, you must have
`manage_data_frame_transforms` cluster privileges to use this API. You must also
have `view_index_metadata` privileges on the source index for the
{dataframe-transform}. For more information, see
{transform}. For more information, see
{stack-ov}/security-privileges.html[Security privileges] and
{stack-ov}/built-in-roles.html[Built-in roles].
[[start-data-frame-transform-desc]]
==== {api-description-title}
When you start a {dataframe-transform}, it creates the destination index if it
When you start a {transform}, it creates the destination index if it
does not already exist. The `number_of_shards` is set to `1` and the
`auto_expand_replicas` is set to `0-1`.
The transform deduces the mapping definitions from the source indices. For
The {transform} deduces the mapping definitions from the source indices. For
scripted fields, it uses <<dynamic-mapping,dynamic mappings>>. If a field in the
destination index is created by `scripted_metric` or `bucket_script`
aggregations, the transform uses dynamic mappings unless a template exists or
aggregations, the {transform} uses dynamic mappings unless a template exists or
the destination index already exists. Mapping definitions in the destination
index take precedence over dynamic mappings and templates.
When the {dataframe-transform} starts, a series of validations occur to ensure
When the {transform} starts, a series of validations occur to ensure
its success. If you deferred validation when you created the
{dataframe-transform}, they occur when you start the transform--with the
{transform}, they occur when you start the {transform}--with the
exception of privilege checks. When {es} {security-features} are enabled, the
{dataframe-transform} remembers which roles the user that created it had at the
{transform} remembers which roles the user that created it had at the
time of creation and uses those same roles. If those roles do not have the
required privileges on the source and destination indices, the
{dataframe-transform} fails when it attempts unauthorized operations.
{transform} fails when it attempts unauthorized operations.
[[start-data-frame-transform-path-parms]]
==== {api-path-parms-title}
`<data_frame_transform_id>`::
(Required, string) Identifier for the {dataframe-transform}. This identifier
(Required, string) Identifier for the {transform}. This identifier
can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and
underscores. It must start and end with alphanumeric characters.
@ -67,7 +67,7 @@ POST _data_frame/transforms/ecommerce_transform/_start
--------------------------------------------------
// TEST[skip:set up kibana samples]
When the {dataframe-transform} starts, you receive the following results:
When the {transform} starts, you receive the following results:
[source,console-result]
----

28
docs/reference/data-frames/apis/stop-transform.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[stop-data-frame-transform]]
=== Stop {dataframe-transforms} API
=== Stop {transforms} API
[subs="attributes"]
++++
<titleabbrev>Stop {dataframe-transforms}</titleabbrev>
<titleabbrev>Stop {transforms}</titleabbrev>
++++
Stops one or more {dataframe-transforms}.
Stops one or more {transforms}.
beta[]
@ -33,16 +33,16 @@ see {stack-ov}/security-privileges.html[Security privileges] and
[[stop-data-frame-transform-desc]]
==== {api-description-title}
You can stop multiple {dataframe-transforms} in a single API request by using a
comma-separated list of {dataframe-transforms} or a wildcard expression.
All {dataframe-transforms} can be stopped by using `_all` or `*` as the
You can stop multiple {transforms} in a single API request by using a
comma-separated list of {transforms} or a wildcard expression.
All {transforms} can be stopped by using `_all` or `*` as the
`<data_frame_transform_id>`.
[[stop-data-frame-transform-path-parms]]
==== {api-path-parms-title}
`<data_frame_transform_id>`::
(Required, string) Identifier for the {dataframe-transform}. This identifier
(Required, string) Identifier for the {transform}. This identifier
can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and
underscores. It must start and end with alphanumeric characters.
@ -53,15 +53,15 @@ All {dataframe-transforms} can be stopped by using `_all` or `*` as the
(Optional, boolean) Specifies what to do when the request:
+
--
* Contains wildcard expressions and there are no {dataframe-transforms} that match.
* Contains wildcard expressions and there are no {transforms} that match.
* Contains the `_all` string or no identifiers and there are no matches.
* Contains wildcard expressions and there are only partial matches.
The default value is `true`, which returns a successful acknowledgement message
when there are no matches. When there are only partial matches, the API stops
the appropriate {dataframe-transforms}. For example, if the request contains
`test-id1*,test-id2*` as the identifiers and there are no {dataframe-transforms}
that match `test-id2*`, the API nonetheless stops the {dataframe-transforms}
the appropriate {transforms}. For example, if the request contains
`test-id1*,test-id2*` as the identifiers and there are no {transforms}
that match `test-id2*`, the API nonetheless stops the {transforms}
that match `test-id1*`.
If this parameter is `false`, the request returns a `404` status code when there
@ -70,10 +70,10 @@ are no matches or only partial matches.
`timeout`::
(Optional, time value) If `wait_for_completion=true`, the API blocks for (at
maximum) the specified duration while waiting for the transform to stop. If
maximum) the specified duration while waiting for the {transform} to stop. If
more than `timeout` time has passed, the API throws a timeout exception. Even
if a timeout exception is thrown, the stop request is still processing and
eventually moves the transform to `STOPPED`. The timeout simply means the API
eventually moves the {transform} to `STOPPED`. The timeout simply means the API
call itself timed out while waiting for the status change. Defaults to `30s`
`wait_for_completion`::
@ -97,7 +97,7 @@ POST _data_frame/transforms/ecommerce_transform/_stop
--------------------------------------------------
// TEST[skip:set up kibana samples]
When the {dataframe-transform} stops, you receive the following results:
When the {transform} stops, you receive the following results:
[source,console-result]
----

34
docs/reference/data-frames/apis/transformresource.asciidoc
View File

@ -1,9 +1,9 @@
[role="xpack"]
[testenv="basic"]
[[data-frame-transform-resource]]
=== {dataframe-transform-cap} resources
=== {transform-cap} resources
{dataframe-transform-cap} resources relate to the <<data-frame-apis>>.
{transform-cap} resources relate to the <<data-frame-apis>>.
For more information, see
{stack-ov}/ecommerce-dataframes.html[Transforming your data with {dataframes}].
@ -13,42 +13,42 @@ For more information, see
==== {api-definitions-title}
`description`::
(string) A description of the {dataframe-transform}.
(string) A description of the {transform}.
`dest`::
(object) The destination for the {dataframe-transform}. See
(object) The destination for the {transform}. See
<<data-frame-transform-dest>>.
`frequency`::
(time units) The interval between checks for changes in the source indices
when the {dataframe-transform} is running continuously. Also determines the
retry interval in the event of transient failures while the {dataframe-transform} is
when the {transform} is running continuously. Also determines the
retry interval in the event of transient failures while the {transform} is
searching or indexing. The minimum value is `1s` and the maximum is `1h`. The
default value is `1m`.
`id`::
(string) A unique identifier for the {dataframe-transform}.
(string) A unique identifier for the {transform}.
`pivot`::
(object) The method for transforming the data. See
<<data-frame-transform-pivot>>.
`source`::
(object) The source of the data for the {dataframe-transform}. See
(object) The source of the data for the {transform}. See
<<data-frame-transform-source>>.
[[data-frame-transform-dest]]
==== Dest objects
{dataframe-transform-cap} resources contain `dest` objects. For example, when
you create a {dataframe-transform}, you must define its destination.
{transform-cap} resources contain `dest` objects. For example, when
you create a {transform}, you must define its destination.
[discrete]
[[data-frame-transform-dest-properties]]
===== {api-definitions-title}
`index`::
(string) The _destination index_ for the {dataframe-transform}.
(string) The _destination index_ for the {transform}.
`pipeline`::
(string) The unique identifier for a <<pipeline,pipeline>>.
@ -56,15 +56,15 @@ you create a {dataframe-transform}, you must define its destination.
[[data-frame-transform-source]]
==== Source objects
{dataframe-transform-cap} resources contain `source` objects. For example, when
you create a {dataframe-transform}, you must define its source.
{transform-cap} resources contain `source` objects. For example, when
you create a {transform}, you must define its source.
[discrete]
[[data-frame-transform-source-properties]]
===== {api-definitions-title}
`index`::
(string or array) The _source indices_ for the {dataframe-transform}. It can
(string or array) The _source indices_ for the {transform}. It can
be a single index, an index pattern (for example, `"myindex*"`), or an array
of indices (for example, `["index1", "index2"]`).
@ -75,7 +75,7 @@ you create a {dataframe-transform}, you must define its source.
[[data-frame-transform-pivot]]
==== Pivot objects
{dataframe-transform-cap} resources contain `pivot` objects, which define the
{transform-cap} resources contain `pivot` objects, which define the
pivot function `group by` fields and the aggregation to reduce the data.
[discrete]
@ -100,7 +100,7 @@ pivot function `group by` fields and the aggregation to reduce the data.
* {ref}/search-aggregations-pipeline-bucket-script-aggregation.html[Bucket Script]
* {ref}/search-aggregations-pipeline-bucket-selector-aggregation.html[Bucket Selector]
IMPORTANT: {dataframe-transforms-cap} support a subset of the functionality in
IMPORTANT: {transforms-cap} support a subset of the functionality in
composite aggregations. See
{stack-ov}/dataframe-limitations.html[{dataframe-cap} limitations].
@ -126,4 +126,4 @@ composite aggregations. See
==== {api-examples-title}
See the
<<put-data-frame-transform-example,create {dataframe-transforms} API examples>>.
<<put-data-frame-transform-example,create {transforms} API examples>>.

34
docs/reference/data-frames/apis/update-transform.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[update-data-frame-transform]]
=== Update {dataframe-transforms} API
=== Update {transforms} API
[subs="attributes"]
++++
<titleabbrev>Update {dataframe-transforms}</titleabbrev>
<titleabbrev>Update {transforms}</titleabbrev>
++++
Updates an existing {dataframe-transform}.
Updates an existing {transform}.
beta[]
@ -31,16 +31,16 @@ information, see {stack-ov}/security-privileges.html[Security privileges] and
[[update-data-frame-transform-desc]]
==== {api-description-title}
This API updates an existing {dataframe-transform}. All settings except description do not
take effect until after the {dataframe-transform} starts the next checkpoint. This is
This API updates an existing {transform}. All settings except description do not
take effect until after the {transform} starts the next checkpoint. This is
so there is consistency with the pivoted data in each checkpoint.
IMPORTANT: When {es} {security-features} are enabled, your {dataframe-transform}
IMPORTANT: When {es} {security-features} are enabled, your {transform}
remembers which roles the user who updated it had at the time of update and
runs with those privileges.
IMPORTANT: You must use {kib} or this API to update a {dataframe-transform}.
Do not update a {dataframe-transform} directly via
IMPORTANT: You must use {kib} or this API to update a {transform}.
Do not update a {transform} directly via
`.data-frame-internal*` indices using the Elasticsearch index API.
If {es} {security-features} are enabled, do not give users any
privileges on `.data-frame-internal*` indices.
@ -49,7 +49,7 @@ IMPORTANT: You must use {kib} or this API to update a {dataframe-transform}.
==== {api-path-parms-title}
`<data_frame_transform_id>`::
(Required, string) Identifier for the {dataframe-transform}. This identifier
(Required, string) Identifier for the {transform}. This identifier
can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and
underscores. It must start and end with alphanumeric characters.
@ -59,29 +59,29 @@ IMPORTANT: You must use {kib} or this API to update a {dataframe-transform}.
`defer_validation`::
(Optional, boolean) When `true`, deferrable validations are not run. This
behavior may be desired if the source index does not exist until after the
{dataframe-transform} is updated.
{transform} is updated.
[[update-data-frame-transform-request-body]]
==== {api-request-body-title}
`description`::
(Optional, string) Free text description of the {dataframe-transform}.
(Optional, string) Free text description of the {transform}.
`dest`::
(Optional, object) The destination configuration, which has the
following properties:
`index`:::
(Required, string) The _destination index_ for the {dataframe-transform}.
(Required, string) The _destination index_ for the {transform}.
`pipeline`:::
(Optional, string) The unique identifier for a <<pipeline,pipeline>>.
`frequency`::
(Optional, <<time-units, time units>>) The interval between checks for changes
in the source indices when the {dataframe-transform} is running continuously.
in the source indices when the {transform} is running continuously.
Also determines the retry interval in the event of transient failures while
the {dataframe-transform} is searching or indexing. The minimum value is `1s`
the {transform} is searching or indexing. The minimum value is `1s`
and the maximum is `1h`. The default value is `1m`.
`source`::
@ -90,7 +90,7 @@ IMPORTANT: You must use {kib} or this API to update a {dataframe-transform}.
`index`:::
(Required, string or array) The _source indices_ for the
{dataframe-transform}. It can be a single index, an index pattern (for
{transform}. It can be a single index, an index pattern (for
example, `"myindex*"`), or an array of indices (for example,
`["index1", "index2"]`).
@ -101,7 +101,7 @@ IMPORTANT: You must use {kib} or this API to update a {dataframe-transform}.
`sync`::
(Optional, object) Defines the properties required to run continuously.
`time`:::
(Required, object) Specifies that the {dataframe-transform} uses a time
(Required, object) Specifies that the {transform} uses a time
field to synchronize the source and destination indices.
`field`::::
(Required, string) The date field that is used to identify new documents
@ -151,7 +151,7 @@ POST _data_frame/transforms/simple-kibana-ecomm-pivot/_update
--------------------------------------------------
// TEST[setup:simple_kibana_continuous_pivot]
When the transform is updated, you receive the updated configuration:
When the {transform} is updated, you receive the updated configuration:
[source,console-result]
----

2
docs/reference/rest-api/defs.asciidoc
View File

@ -8,7 +8,7 @@ These resource definitions are used in APIs related to {ml-features} and
* <<ml-datafeed-resource,{dfeeds-cap}>>
* <<ml-datafeed-counts,{dfeed-cap} counts>>
* <<ml-dfanalytics-resources,{dfanalytics-cap}>>
* <<data-frame-transform-resource,{dataframe-transforms-cap}>>
* <<data-frame-transform-resource,{transforms-cap}>>
* <<ml-evaluate-dfanalytics-resources,Evaluate {dfanalytics}>>
* <<ml-job-resource,{anomaly-jobs-cap}>>
* <<ml-jobstats,{anomaly-jobs-cap} statistics>>

2
docs/reference/rest-api/index.asciidoc
View File

@ -14,7 +14,7 @@ not be included yet.
* <<cat, cat APIs>>
* <<cluster, Cluster APIs>>
* <<ccr-apis,{ccr-cap} APIs>>
* <<data-frame-apis,{dataframe-transform-cap} APIs>>
* <<data-frame-apis,{transform-cap} APIs>>
* <<docs, Document APIs>>
* <<graph-explore-api,Graph Explore API>>
* <<indices, Index APIs>>

24
docs/reference/settings/data-frames-settings.asciidoc
View File

@ -1,13 +1,13 @@
[role="xpack"]
[[data-frames-settings]]
=== {dataframe-transforms-cap} settings in Elasticsearch
=== {transforms-cap} settings in Elasticsearch
[subs="attributes"]
++++
<titleabbrev>{dataframe-transforms-cap} settings</titleabbrev>
<titleabbrev>{transforms-cap} settings</titleabbrev>
++++
You do not need to configure any settings to use {dataframe-transforms}. It is enabled by default.
You do not need to configure any settings to use {transforms}. It is enabled by default.
All of these settings can be added to the `elasticsearch.yml` configuration file.
The dynamic settings can also be updated across a cluster with the
@ -18,23 +18,23 @@ file.
[float]
[[general-data-frames-settings]]
==== General {dataframe-transforms} settings
==== General {transforms} settings
`xpack.data_frame.enabled`::
Set to `true` (default) to enable {dataframe-transforms} on the node. +
Set to `true` (default) to enable {transforms} on the node. +
+
If set to `false` in `elasticsearch.yml`, the {dataframe-transform} APIs are disabled on the node.
Therefore the node cannot start or administrate transforms or receive transport (internal)
communication requests related to {dataframe-transform} APIs.
If set to `false` in `elasticsearch.yml`, the {transform} APIs are disabled on the node.
Therefore the node cannot start or administrate {transform} or receive transport (internal)
communication requests related to {transform} APIs.
+
IMPORTANT: If you want to use {dataframe-transform} features in your cluster, you must have
IMPORTANT: If you want to use {transform} features in your cluster, you must have
`xpack.data_frame.enabled` set to `true` on all master-eligible nodes. This is the
default behavior.
`xpack.data_frame.num_transform_failure_retries` (<<cluster-update-settings,Dynamic>>)::
The number of times that a {dataframe-transform} retries when it experiences a
non-fatal error. Once the number of retries is exhausted, the {dataframe-transform}
The number of times that a {transform} retries when it experiences a
non-fatal error. Once the number of retries is exhausted, the {transform}
task will be marked as `failed`. The default value is `10` with a valid minimum of `0`
and maximum of `100`.
If a {dataframe-transform} is already running, it will have to be restarted
If a {transform} is already running, it will have to be restarted
to use the changed setting.

18
docs/reference/transform/api-quickref.asciidoc
View File

@ -2,7 +2,7 @@
[[df-api-quickref]]
== API quick reference
All {dataframe-transform} endpoints have the following base:
All {transform} endpoints have the following base:
[source,js]
----
@ -10,12 +10,12 @@ All {dataframe-transform} endpoints have the following base:
----
// NOTCONSOLE
* {ref}/put-data-frame-transform.html[Create {dataframe-transforms}]
* {ref}/delete-data-frame-transform.html[Delete {dataframe-transforms}]
* {ref}/get-data-frame-transform.html[Get {dataframe-transforms}]
* {ref}/get-data-frame-transform-stats.html[Get {dataframe-transforms} statistics]
* {ref}/preview-data-frame-transform.html[Preview {dataframe-transforms}]
* {ref}/start-data-frame-transform.html[Start {dataframe-transforms}]
* {ref}/stop-data-frame-transform.html[Stop {dataframe-transforms}]
* {ref}/put-data-frame-transform.html[Create {transforms}]
* {ref}/delete-data-frame-transform.html[Delete {transforms}]
* {ref}/get-data-frame-transform.html[Get {transforms}]
* {ref}/get-data-frame-transform-stats.html[Get {transforms} statistics]
* {ref}/preview-data-frame-transform.html[Preview {transforms}]
* {ref}/start-data-frame-transform.html[Start {transforms}]
* {ref}/stop-data-frame-transform.html[Stop {transforms}]
For the full list, see {ref}/data-frame-apis.html[{dataframe-transform-cap} APIs].
For the full list, see {ref}/data-frame-apis.html[{transform-cap} APIs].

52
docs/reference/transform/checkpoints.asciidoc
View File

@ -1,24 +1,24 @@
[role="xpack"]
[[ml-transform-checkpoints]]
== How {dataframe-transform} checkpoints work
== How {transform} checkpoints work
++++
<titleabbrev>How checkpoints work</titleabbrev>
++++
beta[]
Each time a {dataframe-transform} examines the source indices and creates or
Each time a {transform} examines the source indices and creates or
updates the destination index, it generates a _checkpoint_.
If your {dataframe-transform} runs only once, there is logically only one
checkpoint. If your {dataframe-transform} runs continuously, however, it creates
If your {transform} runs only once, there is logically only one
checkpoint. If your {transform} runs continuously, however, it creates
checkpoints as it ingests and transforms new source data.
To create a checkpoint, the {cdataframe-transform}:
To create a checkpoint, the {ctransform}:
. Checks for changes to source indices.
+
Using a simple periodic timer, the {dataframe-transform} checks for changes to
Using a simple periodic timer, the {transform} checks for changes to
the source indices. This check is done based on the interval defined in the
transform's `frequency` property.
+
@ -27,32 +27,32 @@ then it waits for the next timer.
. Identifies which entities have changed.
+
The {dataframe-transform} searches to see which entities have changed since the
last time it checked. The transform's `sync` configuration object identifies a
time field in the source indices. The transform uses the values in that field to
synchronize the source and destination indices.
The {transform} searches to see which entities have changed since the
last time it checked. The `sync` configuration object in the {transform}
identifies a time field in the source indices. The {transform} uses the values
in that field to synchronize the source and destination indices.
. Updates the destination index (the {dataframe}) with the changed entities.
+
--
The {dataframe-transform} applies changes related to either new or changed
The {transform} applies changes related to either new or changed
entities to the destination index. The set of changed entities is paginated. For
each page, the {dataframe-transform} performs a composite aggregation using a
each page, the {transform} performs a composite aggregation using a
`terms` query. After all the pages of changes have been applied, the checkpoint
is complete.
--
This checkpoint process involves both search and indexing activity on the
cluster. We have attempted to favor control over performance while developing
{dataframe-transforms}. We decided it was preferable for the
{dataframe-transform} to take longer to complete, rather than to finish quickly
{transforms}. We decided it was preferable for the
{transform} to take longer to complete, rather than to finish quickly
and take precedence in resource consumption. That being said, the cluster still
requires enough resources to support both the composite aggregation search and
the indexing of its results.
TIP: If the cluster experiences unsuitable performance degradation due to the
{dataframe-transform}, stop the transform. Consider whether you can apply a
source query to the {dataframe-transform} to reduce the scope of data it
{transform}, stop the {transform}. Consider whether you can apply a
source query to the {transform} to reduce the scope of data it
processes. Also consider whether the cluster has sufficient resources in place
to support both the composite aggregation search and the indexing of its
results.
@ -61,28 +61,28 @@ results.
[[ml-transform-checkpoint-errors]]
==== Error handling
Failures in {dataframe-transforms} tend to be related to searching or indexing.
To increase the resiliency of {dataframe-transforms}, the cursor positions of
Failures in {transforms} tend to be related to searching or indexing.
To increase the resiliency of {transforms}, the cursor positions of
the aggregated search and the changed entities search are tracked in memory and
persisted periodically.
Checkpoint failures can be categorized as follows:
* Temporary failures: The checkpoint is retried. If 10 consecutive failures
occur, the {dataframe-transform} has a failed status. For example, this
occur, the {transform} has a failed status. For example, this
situation might occur when there are shard failures and queries return only
partial results.
* Irrecoverable failures: The {dataframe-transform} immediately fails. For
* Irrecoverable failures: The {transform} immediately fails. For
example, this situation occurs when the source index is not found.
* Adjustment failures: The {dataframe-transform} retries with adjusted settings.
* Adjustment failures: The {transform} retries with adjusted settings.
For example, if a parent circuit breaker memory errors occur during the
composite aggregation, the transform receives partial results. The aggregated
composite aggregation, the {transform} receives partial results. The aggregated
search is retried with a smaller number of buckets. This retry is performed at
the interval defined in the transform's `frequency` property. If the search
is retried to the point where it reaches a minimal number of buckets, an
the interval defined in the `frequency` property for the {transform}. If the
search is retried to the point where it reaches a minimal number of buckets, an
irrecoverable failure occurs.
If the node running the {dataframe-transforms} fails, the transform restarts
If the node running the {transforms} fails, the {transform} restarts
from the most recent persisted cursor position. This recovery process might
repeat some of the work the transform had already done, but it ensures data
repeat some of the work the {transform} had already done, but it ensures data
consistency.

12
docs/reference/transform/dataframe-examples.asciidoc
View File

@ -1,14 +1,14 @@
[role="xpack"]
[testenv="basic"]
[[dataframe-examples]]
== {dataframe-transform-cap} examples
== {transform-cap} examples
++++
<titleabbrev>Examples</titleabbrev>
++++
beta[]
These examples demonstrate how to use {dataframe-transforms} to derive useful
These examples demonstrate how to use {transforms} to derive useful
insights from your data. All the examples use one of the
{kibana-ref}/add-sample-data.html[{kib} sample datasets]. For a more detailed,
step-by-step example, see
@ -67,7 +67,7 @@ gives more context to the final results.
NOTE: In the example above, condensed JSON formatting has been used for easier
readability of the pivot object.
The preview {dataframe-transforms} API enables you to see the layout of the
The preview {transforms} API enables you to see the layout of the
{dataframe} in advance, populated with some sample values. For example:
[source,js]
@ -195,7 +195,7 @@ or flight stats for any of the featured destination or origin airports.
[[example-clientips]]
=== Finding suspicious client IPs by using scripted metrics
With {dataframe-transforms}, you can use
With {transforms}, you can use
{ref}/search-aggregations-metrics-scripted-metric-aggregation.html[scripted
metric aggregations] on your data. These aggregations are flexible and make
it possible to perform very complex processing. Let's use scripted metrics to
@ -278,8 +278,8 @@ POST _data_frame/transforms/_preview
----------------------------------
// TEST[skip:setup kibana sample data]
<1> This range query limits the transform to documents that are within the last
30 days at the point in time the {dataframe-transform} checkpoint is processed.
<1> This range query limits the {transform} to documents that are within the last
30 days at the point in time the {transform} checkpoint is processed.
For batch {dataframes} this occurs once.
<2> This is the destination index for the {dataframe}. It is ignored by
`_preview`.

44
docs/reference/transform/ecommerce-example.asciidoc
View File

@ -5,10 +5,10 @@
beta[]
<<ml-dataframes,{dataframe-transforms-cap}>> enable you to retrieve information
<<ml-dataframes,{transforms-cap}>> enable you to retrieve information
from an {es} index, transform it, and store it in another index. Let's use the
{kibana-ref}/add-sample-data.html[{kib} sample data] to demonstrate how you can
pivot and summarize your data with {dataframe-transforms}.
pivot and summarize your data with {transforms}.
. If the {es} {security-features} are enabled, obtain a user ID with sufficient
@ -16,7 +16,7 @@ privileges to complete these steps.
+
--
You need `manage_data_frame_transforms` cluster privileges to preview and create
{dataframe-transforms}. Members of the built-in `data_frame_transforms_admin`
{transforms}. Members of the built-in `data_frame_transforms_admin`
role have these privileges.
You also need `read` and `view_index_metadata` index privileges on the source
@ -51,10 +51,10 @@ at least one aggregation. You can preview what the transformed data will look
like, so go ahead and play with it!
For example, go to *Machine Learning* > *Data Frames* in {kib} and use the
wizard to create a {dataframe-transform}:
wizard to create a {transform}:
[role="screenshot"]
image::images/ecommerce-pivot1.jpg["Creating a simple {dataframe-transform} in {kib}"]
image::images/ecommerce-pivot1.jpg["Creating a simple {transform} in {kib}"]
In this case, we grouped the data by customer ID and calculated the sum of
products each customer purchased.
@ -71,7 +71,7 @@ We'll accomplish this by using the
on the `order_id` field:
[role="screenshot"]
image::images/ecommerce-pivot2.jpg["Adding multiple aggregations to a {dataframe-transform} in {kib}"]
image::images/ecommerce-pivot2.jpg["Adding multiple aggregations to a {transform} in {kib}"]
TIP: If you're interested in a subset of the data, you can optionally include a
{ref}/search-request-body.html#request-body-search-query[query] element. In this
@ -81,7 +81,7 @@ If you want to use more complex queries, you can create your {dataframe} from a
{kibana-ref}/save-open-search.html[saved search].
If you prefer, you can use the
{ref}/preview-data-frame-transform.html[preview {dataframe-transforms} API]:
{ref}/preview-data-frame-transform.html[preview {transforms} API]:
[source,console]
--------------------------------------------------
@ -134,28 +134,28 @@ POST _data_frame/transforms/_preview
--
. When you are satisfied with what you see in the preview, create the
{dataframe-transform}.
{transform}.
+
--
.. Supply a job ID and the name of the target (or _destination_) index.
.. Decide whether you want the {dataframe-transform} to run once or continuously.
.. Decide whether you want the {transform} to run once or continuously.
--
+
--
Since this sample data index is unchanging, let's use the default behavior and
just run the {dataframe-transform} once.
just run the {transform} once.
[role="screenshot"]
image::images/ecommerce-batch.jpg["Specifying the {dataframe-transform} options in {kib}"]
image::images/ecommerce-batch.jpg["Specifying the {transform} options in {kib}"]
If you want to try it out, however, go ahead and click on *Continuous mode*.
You must choose a field that the {dataframe-transform} can use to check which
You must choose a field that the {transform} can use to check which
entities have changed. In general, it's a good idea to use the ingest timestamp
field. In this example, however, you can use the `order_date` field.
If you prefer, you can use the
{ref}/put-data-frame-transform.html[create {dataframe-transforms} API]. For
{ref}/put-data-frame-transform.html[create {transforms} API]. For
example:
[source,console]
@ -215,23 +215,23 @@ PUT _data_frame/transforms/ecommerce-customer-transform
// TEST[skip:setup kibana sample data]
--
. Start the {dataframe-transform}.
. Start the {transform}.
+
--
TIP: Even though resource utilization is automatically adjusted based on the
cluster load, a {dataframe-transform} increases search and indexing load on your
cluster load, a {transform} increases search and indexing load on your
cluster while it runs. If you're experiencing an excessive load, however, you
can stop it.
You can start, stop, and manage {dataframe-transforms} in {kib}:
You can start, stop, and manage {transforms} in {kib}:
[role="screenshot"]
image::images/dataframe-transforms.jpg["Managing {dataframe-transforms} in {kib}"]
image::images/dataframe-transforms.jpg["Managing {transforms} in {kib}"]
Alternatively, you can use the
{ref}/start-data-frame-transform.html[start {dataframe-transforms}] and
{ref}/stop-data-frame-transform.html[stop {dataframe-transforms}] APIs. For
{ref}/start-data-frame-transform.html[start {transforms}] and
{ref}/stop-data-frame-transform.html[stop {transforms}] APIs. For
example:
[source,console]
@ -252,8 +252,8 @@ image::images/ecommerce-results.jpg["Exploring the new index in {kib}"]
--
TIP: If you do not want to keep the {dataframe-transform}, you can delete it in
TIP: If you do not want to keep the {transform}, you can delete it in
{kib} or use the
{ref}/delete-data-frame-transform.html[delete {dataframe-transform} API]. When
you delete a {dataframe-transform}, its destination index and {kib} index
{ref}/delete-data-frame-transform.html[delete {transform} API]. When
you delete a {transform}, its destination index and {kib} index
patterns remain.

4
docs/reference/transform/index.asciidoc
View File

@ -1,6 +1,6 @@
[role="xpack"]
[[ml-dataframes]]
= {dataframe-transforms-cap}
= {transforms-cap}
[partintro]
--
@ -65,7 +65,7 @@ create a {dataframe} to cache results. Thus, each user doesn't need to run the
aggregation query.
Though there are multiple ways to create {dataframes}, this content pertains
to one specific method: _{dataframe-transforms}_.
to one specific method: _{transforms}_.
* <<ml-transform-overview>>
* <<df-api-quickref>>

74
docs/reference/transform/limitations.asciidoc
View File

@ -1,6 +1,6 @@
[role="xpack"]
[[dataframe-limitations]]
== {dataframe-transform-cap} limitations
== {transform-cap} limitations
[subs="attributes"]
++++
<titleabbrev>Limitations</titleabbrev>
@ -13,14 +13,14 @@ the Elastic {dataframe} feature:
[float]
[[df-compatibility-limitations]]
=== Beta {dataframe-transforms} do not have guaranteed backwards or forwards compatibility
=== Beta {transforms} do not have guaranteed backwards or forwards compatibility
Whilst {dataframe-transforms} are beta, it is not guaranteed that a
{dataframe-transform} created in a previous version of the {stack} will be able
Whilst {transforms} are beta, it is not guaranteed that a
{transform} created in a previous version of the {stack} will be able
to start and operate in a future version. Neither can support be provided for
{dataframe-transform} tasks to be able to operate in a cluster with mixed node
{transform} tasks to be able to operate in a cluster with mixed node
versions.
Please note that the output of a {dataframe-transform} is persisted to a
Please note that the output of a {transform} is persisted to a
destination index. This is a normal {es} index and is not affected by the beta
status.
@ -29,7 +29,7 @@ status.
=== {dataframe-cap} UI will not work during a rolling upgrade from 7.2
If your cluster contains mixed version nodes, for example during a rolling
upgrade from 7.2 to a newer version, and {dataframe-transforms} have been
upgrade from 7.2 to a newer version, and {transforms} have been
created in 7.2, the {dataframe} UI will not work. Please wait until all nodes
have been upgraded to the newer version before using the {dataframe} UI.
@ -46,23 +46,23 @@ table.
[[df-ccs-limitations]]
=== {ccs-cap} is not supported
{ccs-cap} is not supported for {dataframe-transforms}.
{ccs-cap} is not supported for {transforms}.
[float]
[[df-kibana-limitations]]
=== Up to 1,000 {dataframe-transforms} are supported
=== Up to 1,000 {transforms} are supported
A single cluster will support up to 1,000 {dataframe-transforms}.
A single cluster will support up to 1,000 {transforms}.
When using the
{ref}/get-data-frame-transform.html[GET {dataframe-transforms} API] a total
`count` of transforms is returned. Use the `size` and `from` parameters to
{ref}/get-data-frame-transform.html[GET {transforms} API] a total
`count` of {transforms} is returned. Use the `size` and `from` parameters to
enumerate through the full list.
[float]
[[df-aggresponse-limitations]]
=== Aggregation responses may be incompatible with destination index mappings
When a {dataframe-transform} is first started, it will deduce the mappings
When a {transform} is first started, it will deduce the mappings
required for the destination index. This process is based on the field types of
the source index and the aggregations used. If the fields are derived from
{ref}/search-aggregations-metrics-scripted-metric-aggregation.html[`scripted_metrics`]
@ -72,15 +72,15 @@ deduced mappings may be incompatible with the actual data. For example, numeric
overflows might occur or dynamically mapped fields might contain both numbers
and strings. Please check {es} logs if you think this may have occurred. As a
workaround, you may define custom mappings prior to starting the
{dataframe-transform}. For example,
{transform}. For example,
{ref}/indices-create-index.html[create a custom destination index] or
{ref}/indices-templates.html[define an index template].
[float]
[[df-batch-limitations]]
=== Batch {dataframe-transforms} may not account for changed documents
=== Batch {transforms} may not account for changed documents
A batch {dataframe-transform} uses a
A batch {transform} uses a
{ref}/search-aggregations-bucket-composite-aggregation.html[composite aggregation]
which allows efficient pagination through all buckets. Composite aggregations
do not yet support a search context, therefore if the source data is changed
@ -91,8 +91,8 @@ results may not include these changes.
[[df-consistency-limitations]]
=== {cdataframe-cap} consistency does not account for deleted or updated documents
While the process for {cdataframe-transforms} allows the continual recalculation
of the {dataframe-transform} as new data is being ingested, it does also have
While the process for {transforms} allows the continual recalculation
of the {transform} as new data is being ingested, it does also have
some limitations.
Changed entities will only be identified if their time field
@ -106,7 +106,7 @@ composite aggregation performed in consecutive checkpoint processing will search
over different source data, and entities that only existed in the deleted index
will not be removed from the {dataframe} destination index.
Depending on your use case, you may wish to recreate the {dataframe-transform}
Depending on your use case, you may wish to recreate the {transform}
entirely after deletions. Alternatively, if your use case is tolerant to
historical archiving, you may wish to include a max ingest timestamp in your
aggregation. This will allow you to exclude results that have not been recently
@ -115,9 +115,9 @@ updated when viewing the {dataframe} destination index.
[float]
[[df-deletion-limitations]]
=== Deleting a {dataframe-transform} does not delete the {dataframe} destination index or {kib} index pattern
=== Deleting a {transform} does not delete the {dataframe} destination index or {kib} index pattern
When deleting a {dataframe-transform} using `DELETE _data_frame/transforms/index`
When deleting a {transform} using `DELETE _data_frame/transforms/index`
neither the {dataframe} destination index nor the {kib} index pattern, should
one have been created, are deleted. These objects must be deleted separately.
@ -125,9 +125,9 @@ one have been created, are deleted. These objects must be deleted separately.
[[df-aggregation-page-limitations]]
=== Handling dynamic adjustment of aggregation page size
During the development of {dataframe-transforms}, control was favoured over
During the development of {transforms}, control was favoured over
performance. In the design considerations, it is preferred for the
{dataframe-transform} to take longer to complete quietly in the background
{transform} to take longer to complete quietly in the background
rather than to finish quickly and take precedence in resource consumption.
Composite aggregations are well suited for high cardinality data enabling
@ -135,21 +135,21 @@ pagination through results. If a {ref}/circuit-breaker.html[circuit breaker]
memory exception occurs when performing the composite aggregated search then we
try again reducing the number of buckets requested. This circuit breaker is
calculated based upon all activity within the cluster, not just activity from
{dataframe-transforms}, so it therefore may only be a temporary resource
{transforms}, so it therefore may only be a temporary resource
availability issue.
For a batch {dataframe-transform}, the number of buckets requested is only ever
For a batch {transform}, the number of buckets requested is only ever
adjusted downwards. The lowering of value may result in a longer duration for the
transform checkpoint to complete. For {cdataframes}, the number of
{transform} checkpoint to complete. For {cdataframes}, the number of
buckets requested is reset back to its default at the start of every checkpoint
and it is possible for circuit breaker exceptions to occur repeatedly in the
{es} logs.
The {dataframe-transform} retrieves data in batches which means it calculates
The {transform} retrieves data in batches which means it calculates
several buckets at once. Per default this is 500 buckets per search/index
operation. The default can be changed using `max_page_search_size` and the
minimum value is 10. If failures still occur once the number of buckets
requested has been reduced to its minimum, then the {dataframe-transform} will
requested has been reduced to its minimum, then the {transform} will
be set to a failed state.
[float]
@ -158,7 +158,7 @@ be set to a failed state.
For each checkpoint, entities are identified that have changed since the last
time the check was performed. This list of changed entities is supplied as a
{ref}/query-dsl-terms-query.html[terms query] to the {dataframe-transform}
{ref}/query-dsl-terms-query.html[terms query] to the {transform}
composite aggregation, one page at a time. Then updates are applied to the
destination index for each page of entities.
@ -170,10 +170,10 @@ The index setting
{ref}/index-modules.html#dynamic-index-settings[`index.max_terms_count`] defines
the maximum number of terms that can be used in a terms query. The default value
is 65536. If `max_page_search_size` exceeds `index.max_terms_count` the
transform will fail.
{transform} will fail.
Using smaller values for `max_page_search_size` may result in a longer duration
for the transform checkpoint to complete.
for the {transform} checkpoint to complete.
[float]
[[df-scheduling-limitations]]
@ -183,22 +183,22 @@ A {cdataframe} periodically checks for changes to source data. The functionality
of the scheduler is currently limited to a basic periodic timer which can be
within the `frequency` range from 1s to 1h. The default is 1m. This is designed
to run little and often. When choosing a `frequency` for this timer consider
your ingest rate along with the impact that the {dataframe-transform}
your ingest rate along with the impact that the {transform}
search/index operations has other users in your cluster. Also note that retries
occur at `frequency` interval.
[float]
[[df-failed-limitations]]
=== Handling of failed {dataframe-transforms}
=== Handling of failed {transforms}
Failed {dataframe-transforms} remain as a persistent task and should be handled
Failed {transforms} remain as a persistent task and should be handled
appropriately, either by deleting it or by resolving the root cause of the
failure and re-starting.
When using the API to delete a failed {dataframe-transform}, first stop it using
When using the API to delete a failed {transform}, first stop it using
`_stop?force=true`, then delete it.
If starting a failed {dataframe-transform}, after the root cause has been
If starting a failed {transform}, after the root cause has been
resolved, the `_start?force=true` parameter must be specified.
[float]
@ -208,7 +208,7 @@ resolved, the `_start?force=true` parameter must be specified.
After a document is indexed, there is a very small delay until it is available
to search.
A {cdataframe-transform} periodically checks for changed entities between the
A {ctransform} periodically checks for changed entities between the
time since it last checked and `now` minus `sync.time.delay`. This time window
moves without overlapping. If the timestamp of a recently indexed document falls
within this time window but this document is not yet available to search then

24
docs/reference/transform/overview.asciidoc
View File

@ -1,6 +1,6 @@
[role="xpack"]
[[ml-transform-overview]]
== {dataframe-transform-cap} overview
== {transform-cap} overview
++++
<titleabbrev>Overview</titleabbrev>
++++
@ -19,8 +19,8 @@ you to summarize this data, bringing it into an organized, more
analysis-friendly format. For example, you can summarize all the purchases of a
single customer.
You can create {dataframes} by using {dataframe-transforms}.
{dataframe-transforms-cap} enable you to define a pivot, which is a set of
You can create {dataframes} by using {transforms}.
{transforms-cap} enable you to define a pivot, which is a set of
features that transform the index into a different, more digestible format.
Pivoting results in a summary of your data, which is the {dataframe}.
@ -33,19 +33,19 @@ The second step is deciding how you want to aggregate the grouped data. When
using aggregations, you practically ask questions about the index. There are
different types of aggregations, each with its own purpose and output. To learn
more about the supported aggregations and group-by fields, see
{ref}/data-frame-transform-resource.html[{dataframe-transform-cap} resources].
{ref}/data-frame-transform-resource.html[{transform-cap} resources].
As an optional step, you can also add a query to further limit the scope of the
aggregation.
The {dataframe-transform} performs a composite aggregation that
The {transform} performs a composite aggregation that
paginates through all the data defined by the source index query. The output of
the aggregation is stored in a destination index. Each time the
{dataframe-transform} queries the source index, it creates a _checkpoint_. You
can decide whether you want the {dataframe-transform} to run once (batch
{dataframe-transform}) or continuously ({cdataframe-transform}). A batch
{dataframe-transform} is a single operation that has a single checkpoint.
{cdataframe-transforms-cap} continually increment and process checkpoints as new
{transform} queries the source index, it creates a _checkpoint_. You
can decide whether you want the {transform} to run once (batch
{transform}) or continuously ({transform}). A batch
{transform} is a single operation that has a single checkpoint.
{ctransforms-cap} continually increment and process checkpoints as new
source data is ingested.
.Example
@ -57,7 +57,7 @@ information (name, gender, location, etc). Your dataset contains all the transac
from last year.
If you want to check the sales in the different categories in your last fiscal
year, define a {dataframe-transform} that groups the data by the product
year, define a {transform} that groups the data by the product
categories (women's shoes, men's clothing, etc.) and the order date. Use the
last year as the interval for the order date. Then add a sum aggregation on the
ordered quantity. The result is a {dataframe} that shows the number of sold
@ -66,6 +66,6 @@ items in every product category in the last year.
[role="screenshot"]
image::images/ml-dataframepivot.jpg["Example of a data frame pivot in {kib}"]
IMPORTANT: The {dataframe-transform} leaves your source index intact. It
IMPORTANT: The {transform} leaves your source index intact. It
creates a new index that is dedicated to the {dataframe}.

16
docs/reference/transform/troubleshooting.asciidoc
View File

@ -1,5 +1,5 @@
[[dataframe-troubleshooting]]
== Troubleshooting {dataframe-transforms}
== Troubleshooting {transforms}
[subs="attributes"]
++++
<titleabbrev>Troubleshooting</titleabbrev>
@ -9,21 +9,21 @@ Use the information in this section to troubleshoot common problems.
include::{stack-repo-dir}/help.asciidoc[tag=get-help]
If you encounter problems with your {dataframe-transforms}, you can gather more
If you encounter problems with your {transforms}, you can gather more
information from the following files and APIs:
* Lightweight audit messages are stored in `.data-frame-notifications-*`. Search
by your `transform_id`.
* The
{ref}/get-data-frame-transform-stats.html[get {dataframe-transform} statistics API]
provides information about the transform status and failures.
* If the {dataframe-transform} exists as a task, you can use the
{ref}/get-data-frame-transform-stats.html[get {transform} statistics API]
provides information about the {transform} status and failures.
* If the {transform} exists as a task, you can use the
{ref}/tasks.html[task management API] to gather task information. For example:
`GET _tasks?actions=data_frame/transforms*&detailed`. Typically, the task exists
when the transform is in a started or failed state.
* The {es} logs from the node that was running the {dataframe-transform} might
when the {transform} is in a started or failed state.
* The {es} logs from the node that was running the {transform} might
also contain useful information. You can identify the node from the notification
messages. Alternatively, if the task still exists, you can get that information
from the get {dataframe-transform} statistics API. For more information, see
from the get {transform} statistics API. For more information, see
{ref}/logging.html[Logging configuration].