[ML] adding delayed_data_check_config to datafeed update docs (#42095) (#42626)

* [ML] adding delayed_data_check_config to datafeed update docs

* [DOCS] Edits delayed data configuration details
This commit is contained in:
Benjamin Trent 2019-05-28 11:36:30 -04:00 committed by GitHub
parent 130c832e10
commit d06618a70d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 29 additions and 19 deletions

View File

@ -61,12 +61,12 @@ A {dfeed} resource has the following properties:
`delayed_data_check_config`::
(object) Specifies whether the data feed checks for missing data and
and the size of the window. For example:
the size of the window. For example:
`{"enabled": true, "check_window": "1h"}` See
<<ml-datafeed-delayed-data-check-config>>.
[[ml-datafeed-chunking-config]]
==== Chunking Configuration Objects
==== Chunking configuration objects
{dfeeds-cap} might be required to search over long time periods, for several months
or years. This search is split into time chunks in order to ensure the load
@ -88,31 +88,33 @@ A chunking configuration object has the following properties:
For example: `3h`.
[[ml-datafeed-delayed-data-check-config]]
==== Delayed Data Check Configuration Objects
==== Delayed data check configuration objects
The {dfeed} can optionally search over indices that have already been read in
an effort to find if any data has since been added to the index. If missing data
is found, it is a good indication that the `query_delay` option is set too low and
the data is being indexed after the {dfeed} has passed that moment in time. See
an effort to determine whether any data has subsequently been added to the index.
If missing data is found, it is a good indication that the `query_delay` option
is set too low and the data is being indexed after the {dfeed} has passed that
moment in time. See
{stack-ov}/ml-delayed-data-detection.html[Working with delayed data].
This check only runs on real-time {dfeeds}
This check runs only on real-time {dfeeds}.
The configuration object has the following properties:
`enabled`::
(boolean) Should the {dfeed} periodically check for data being indexed after reading.
Defaults to `true`
(boolean) Specifies whether the {dfeed} periodically checks for delayed data.
Defaults to `true`.
`check_window`::
(time units) The window of time before the latest finalized bucket that should be searched
for late data. Defaults to `null` which causes an appropriate `check_window` to be calculated
when the real-time {dfeed} runs.
The default `check_window` span calculation is the max between `2h` or `8 * bucket_span`.
(time units) The window of time that is searched for late data. This window of
time ends with the latest finalized bucket. It defaults to `null`, which
causes an appropriate `check_window` to be calculated when the real-time
{dfeed} runs. In particular, the default `check_window` span calculation is
based on the maximum of `2h` or `8 * bucket_span`.
[float]
[[ml-datafeed-counts]]
==== {dfeed-cap} Counts
==== {dfeed-cap} counts
The get {dfeed} statistics API provides information about the operational
progress of a {dfeed}. All of these properties are informational; you cannot

View File

@ -45,6 +45,11 @@ IMPORTANT: You must use {kib} or this API to create a {dfeed}. Do not put a {df
(object) Specifies how data searches are split into time chunks.
See <<ml-datafeed-chunking-config>>.
`delayed_data_check_config`::
(object) Specifies whether the data feed checks for missing data and
the size of the window. See
<<ml-datafeed-delayed-data-check-config>>.
`frequency`::
(time units) The interval at which scheduled queries are made while the {dfeed}
runs in real time. The default value is either the bucket span for short
@ -82,10 +87,6 @@ IMPORTANT: You must use {kib} or this API to create a {dfeed}. Do not put a {df
(unsigned integer) The `size` parameter that is used in {es} searches.
The default value is `1000`.
`delayed_data_check_config`::
(object) Specifies if and with how large a window should the data feed check
for missing data. See <<ml-datafeed-delayed-data-check-config>>.
For more information about these properties,
see <<ml-datafeed-resource>>.

View File

@ -14,7 +14,10 @@ Updates certain properties of a {dfeed}.
`POST _ml/datafeeds/<feed_id>/_update`
//===== Description
===== Description
NOTE: If you update the `delayed_data_check_config` property, you must stop and
start the {dfeed} for the change to be applied.
==== Path Parameters
@ -32,6 +35,10 @@ The following properties can be updated after the {dfeed} is created:
`chunking_config`::
(object) Specifies how data searches are split into time chunks.
See <<ml-datafeed-chunking-config>>.
`delayed_data_check_config`::
(object) Specifies whether the data feed checks for missing data and
the size of the window. See <<ml-datafeed-delayed-data-check-config>>.
`frequency`::
(time units) The interval at which scheduled queries are made while the