2017-06-19 21:23:58 -04:00
|
|
|
[role="xpack"]
|
2018-08-31 19:49:24 -04:00
|
|
|
[testenv="platinum"]
|
2017-04-04 18:26:39 -04:00
|
|
|
[[ml-put-datafeed]]
|
2020-07-20 16:10:54 -04:00
|
|
|
= Create {dfeeds} API
|
2019-04-30 13:46:13 -04:00
|
|
|
|
|
|
|
[subs="attributes"]
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2018-12-20 13:23:28 -05:00
|
|
|
<titleabbrev>Create {dfeeds}</titleabbrev>
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-06-13 16:37:35 -04:00
|
|
|
Instantiates a {dfeed}.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-put-datafeed-request]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-request-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-12-07 15:34:11 -05:00
|
|
|
`PUT _ml/datafeeds/<feed_id>`
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 16:58:42 -04:00
|
|
|
[[ml-put-datafeed-prereqs]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-prereq-title}
|
2019-06-27 16:58:42 -04:00
|
|
|
|
2019-07-26 13:47:03 -04:00
|
|
|
* You must create an {anomaly-job} before you create a {dfeed}.
|
2019-06-27 16:58:42 -04:00
|
|
|
* If {es} {security-features} are enabled, you must have `manage_ml` or `manage`
|
|
|
|
cluster privileges to use this API. See
|
2019-10-07 18:23:19 -04:00
|
|
|
<<security-privileges>>.
|
2019-06-27 16:58:42 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-put-datafeed-desc]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-description-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2020-01-22 11:08:31 -05:00
|
|
|
{ml-docs}/ml-dfeeds.html[{dfeeds-cap}] retrieve data from {es} for analysis by
|
|
|
|
an {anomaly-job}. You can associate only one {dfeed} to each {anomaly-job}.
|
|
|
|
|
|
|
|
The {dfeed} contains a query that runs at a defined interval (`frequency`). If
|
|
|
|
you are concerned about delayed data, you can add a delay (`query_delay`) at
|
|
|
|
each interval. See {ml-docs}/ml-delayed-data-detection.html[Handling delayed data].
|
2017-04-10 11:59:27 -04:00
|
|
|
|
2019-06-27 16:58:42 -04:00
|
|
|
[IMPORTANT]
|
|
|
|
====
|
|
|
|
* You must use {kib} or this API to create a {dfeed}. Do not put a
|
|
|
|
{dfeed} directly to the `.ml-config` index using the {es} index API. If {es}
|
|
|
|
{security-features} are enabled, do not give users `write` privileges on the
|
|
|
|
`.ml-config` index.
|
|
|
|
* When {es} {security-features} are enabled, your {dfeed} remembers which roles
|
|
|
|
the user who created it had at the time of creation and runs the query using
|
2020-04-29 19:29:38 -04:00
|
|
|
those same roles. If you provide
|
|
|
|
<<http-clients-secondary-authorization,secondary authorization headers>>, those
|
|
|
|
credentials are used instead.
|
2019-06-27 16:58:42 -04:00
|
|
|
====
|
2019-02-08 06:35:37 -05:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-put-datafeed-path-parms]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-path-parms-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`<feed_id>`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Required, string)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=datafeed-id]
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
[role="child_attributes"]
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-put-datafeed-request-body]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-request-body-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`aggregations`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, object)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=aggregations]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`chunking_config`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, object)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=chunking-config]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`delayed_data_check_config`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, object)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=delayed-data-check-config]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`frequency`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, <<time-units, time units>>)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=frequency]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`indices`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Required, array)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=indices]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`job_id`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Required, string)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
|
2019-12-30 12:35:16 -05:00
|
|
|
|
|
|
|
`max_empty_searches`::
|
|
|
|
(Optional,integer)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=max-empty-searches]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`query`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, object)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=query]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`query_delay`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, <<time-units, time units>>)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=query-delay]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`script_fields`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, object)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=script-fields]
|
2019-07-12 11:26:31 -04:00
|
|
|
|
|
|
|
`scroll_size`::
|
2019-12-30 12:35:16 -05:00
|
|
|
(Optional, unsigned integer)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=scroll-size]
|
2017-04-10 11:59:27 -04:00
|
|
|
|
2020-02-27 13:43:25 -05:00
|
|
|
`indices_options`::
|
|
|
|
(Optional, object)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=indices-options]
|
2020-02-27 13:43:25 -05:00
|
|
|
|
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-put-datafeed-example]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-examples-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
[source,console]
|
2017-04-04 18:26:39 -04:00
|
|
|
--------------------------------------------------
|
2018-12-07 15:34:11 -05:00
|
|
|
PUT _ml/datafeeds/datafeed-total-requests
|
2017-04-04 18:26:39 -04:00
|
|
|
{
|
2017-10-30 10:54:33 -04:00
|
|
|
"job_id": "total-requests",
|
2018-03-07 16:53:49 -05:00
|
|
|
"indices": ["server-metrics"]
|
2017-04-04 18:26:39 -04:00
|
|
|
}
|
|
|
|
--------------------------------------------------
|
2018-08-31 19:49:24 -04:00
|
|
|
// TEST[skip:setup:server_metrics_job]
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2017-05-02 15:45:42 -04:00
|
|
|
When the {dfeed} is created, you receive the following results:
|
2019-09-06 16:09:09 -04:00
|
|
|
|
|
|
|
[source,console-result]
|
2017-04-04 18:26:39 -04:00
|
|
|
----
|
|
|
|
{
|
2017-10-30 10:54:33 -04:00
|
|
|
"datafeed_id": "datafeed-total-requests",
|
|
|
|
"job_id": "total-requests",
|
|
|
|
"query_delay": "83474ms",
|
2017-05-04 06:37:12 -04:00
|
|
|
"indices": [
|
2017-10-30 10:54:33 -04:00
|
|
|
"server-metrics"
|
2017-04-10 11:59:27 -04:00
|
|
|
],
|
|
|
|
"query": {
|
|
|
|
"match_all": {
|
2017-10-30 10:54:33 -04:00
|
|
|
"boost": 1.0
|
2017-04-10 11:59:27 -04:00
|
|
|
}
|
2017-04-04 18:26:39 -04:00
|
|
|
},
|
2017-04-11 21:52:47 -04:00
|
|
|
"scroll_size": 1000,
|
|
|
|
"chunking_config": {
|
|
|
|
"mode": "auto"
|
|
|
|
}
|
2017-04-04 18:26:39 -04:00
|
|
|
}
|
|
|
|
----
|
2018-02-09 12:16:24 -05:00
|
|
|
// TESTRESPONSE[s/"query_delay": "83474ms"/"query_delay": $body.query_delay/]
|
2018-12-07 15:34:11 -05:00
|
|
|
// TESTRESPONSE[s/"query.boost": "1.0"/"query.boost": $body.query.boost/]
|