2017-06-19 18:23:58 -07:00
|
|
|
[role="xpack"]
|
2018-08-31 16:49:24 -07:00
|
|
|
[testenv="platinum"]
|
2017-04-04 15:26:39 -07:00
|
|
|
[[ml-put-datafeed]]
|
2018-12-20 10:23:28 -08:00
|
|
|
=== Create {dfeeds} API
|
2017-12-14 10:52:49 -08:00
|
|
|
++++
|
2018-12-20 10:23:28 -08:00
|
|
|
<titleabbrev>Create {dfeeds}</titleabbrev>
|
2017-12-14 10:52:49 -08:00
|
|
|
++++
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2018-06-13 13:37:35 -07:00
|
|
|
Instantiates a {dfeed}.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-04-25 14:08:29 -07:00
|
|
|
|
2017-06-06 13:42:47 -07:00
|
|
|
==== Request
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2018-12-07 20:34:11 +00:00
|
|
|
`PUT _ml/datafeeds/<feed_id>`
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-04-25 14:08:29 -07:00
|
|
|
|
2017-06-06 13:42:47 -07:00
|
|
|
==== Description
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-05-02 12:45:42 -07:00
|
|
|
You must create a job before you create a {dfeed}. You can associate only one
|
|
|
|
{dfeed} to each job.
|
2017-04-10 08:59:27 -07:00
|
|
|
|
2017-04-18 15:13:21 -07:00
|
|
|
|
2017-06-06 13:42:47 -07:00
|
|
|
==== Path Parameters
|
2017-04-04 15:26:39 -07:00
|
|
|
|
|
|
|
`feed_id` (required)::
|
2017-05-02 12:45:42 -07:00
|
|
|
(string) A numerical character string that uniquely identifies the {dfeed}.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-04-25 14:08:29 -07:00
|
|
|
|
2017-06-06 13:42:47 -07:00
|
|
|
==== Request Body
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-04-11 18:52:47 -07:00
|
|
|
`aggregations`::
|
2017-05-02 12:45:42 -07:00
|
|
|
(object) If set, the {dfeed} performs aggregation searches.
|
2017-04-26 11:18:51 -07:00
|
|
|
For more information, see <<ml-datafeed-resource>>.
|
2017-04-11 18:52:47 -07:00
|
|
|
|
|
|
|
`chunking_config`::
|
2017-04-27 11:17:06 -07:00
|
|
|
(object) Specifies how data searches are split into time chunks.
|
|
|
|
See <<ml-datafeed-chunking-config>>.
|
2017-04-10 08:59:27 -07:00
|
|
|
|
2017-04-11 18:52:47 -07:00
|
|
|
`frequency`::
|
2017-05-02 12:45:42 -07:00
|
|
|
(time units) The interval at which scheduled queries are made while the {dfeed}
|
|
|
|
runs in real time. The default value is either the bucket span for short
|
2017-04-26 11:18:51 -07:00
|
|
|
bucket spans, or, for longer bucket spans, a sensible fraction of the bucket
|
2017-05-15 09:30:30 -07:00
|
|
|
span. For example: `150s`.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-05-04 11:37:12 +01:00
|
|
|
`indices` (required)::
|
2017-05-15 09:30:30 -07:00
|
|
|
(array) An array of index names. Wildcards are supported. For example:
|
|
|
|
`["it_ops_metrics", "server*"]`.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-04-11 18:52:47 -07:00
|
|
|
`job_id` (required)::
|
2017-04-11 19:26:18 -07:00
|
|
|
(string) A numerical character string that uniquely identifies the job.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-04-11 18:52:47 -07:00
|
|
|
`query`::
|
2017-04-27 11:17:06 -07:00
|
|
|
(object) The {es} query domain-specific language (DSL). This value
|
|
|
|
corresponds to the query object in an {es} search POST body. All the
|
|
|
|
options that are supported by {Es} can be used, as this object is
|
|
|
|
passed verbatim to {es}. By default, this property has the following
|
|
|
|
value: `{"match_all": {"boost": 1}}`.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-04-11 18:52:47 -07:00
|
|
|
`query_delay`::
|
2017-04-27 11:17:06 -07:00
|
|
|
(time units) The number of seconds behind real time that data is queried. For
|
|
|
|
example, if data from 10:04 a.m. might not be searchable in {es} until
|
|
|
|
10:06 a.m., set this property to 120 seconds. The default value is `60s`.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-05-15 09:30:30 -07:00
|
|
|
`script_fields`::
|
|
|
|
(object) Specifies scripts that evaluate custom expressions and returns
|
|
|
|
script fields to the {dfeed}.
|
|
|
|
The <<ml-detectorconfig,detector configuration objects>> in a job can contain
|
2017-06-06 13:42:47 -07:00
|
|
|
functions that use these script fields.
|
2017-05-15 09:30:30 -07:00
|
|
|
For more information,
|
|
|
|
see {ref}/search-request-script-fields.html[Script Fields].
|
|
|
|
|
2017-04-11 18:52:47 -07:00
|
|
|
`scroll_size`::
|
2017-04-27 11:17:06 -07:00
|
|
|
(unsigned integer) The `size` parameter that is used in {es} searches.
|
2017-04-26 11:18:51 -07:00
|
|
|
The default value is `1000`.
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2018-11-15 13:32:45 -06:00
|
|
|
`delayed_data_check_config`::
|
|
|
|
(object) Specifies if and with how large a window should the data feed check
|
|
|
|
for missing data. See <<ml-datafeed-delayed-data-check-config>>.
|
|
|
|
|
2017-04-10 08:59:27 -07:00
|
|
|
For more information about these properties,
|
2017-05-02 12:45:42 -07:00
|
|
|
see <<ml-datafeed-resource>>.
|
2017-04-10 08:59:27 -07:00
|
|
|
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-06-06 13:42:47 -07:00
|
|
|
==== Authorization
|
2017-04-25 14:08:29 -07:00
|
|
|
|
2018-12-19 14:53:37 -08:00
|
|
|
If {es} {security-features} are enabled, you must have `manage_ml`, or `manage`
|
|
|
|
cluster privileges to use this API. For more information, see
|
|
|
|
{stack-ov}/security-privileges.html[Security Privileges].
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-12-15 14:41:10 +00:00
|
|
|
|
2018-12-19 14:53:37 -08:00
|
|
|
==== Security integration
|
2017-12-15 14:41:10 +00:00
|
|
|
|
2018-12-19 14:53:37 -08:00
|
|
|
When {es} {security-features} are enabled, your {dfeed} remembers which roles the
|
|
|
|
user who created it had at the time of creation and runs the query using those
|
|
|
|
same roles.
|
2017-12-15 14:41:10 +00:00
|
|
|
|
|
|
|
|
2017-06-06 13:42:47 -07:00
|
|
|
==== Examples
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-10-30 07:54:33 -07:00
|
|
|
The following example creates the `datafeed-total-requests` {dfeed}:
|
2017-04-04 15:26:39 -07:00
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2018-12-07 20:34:11 +00:00
|
|
|
PUT _ml/datafeeds/datafeed-total-requests
|
2017-04-04 15:26:39 -07:00
|
|
|
{
|
2017-10-30 07:54:33 -07:00
|
|
|
"job_id": "total-requests",
|
2018-03-07 13:53:49 -08:00
|
|
|
"indices": ["server-metrics"]
|
2017-04-04 15:26:39 -07:00
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
// CONSOLE
|
2018-08-31 16:49:24 -07:00
|
|
|
// TEST[skip:setup:server_metrics_job]
|
2017-04-04 15:26:39 -07:00
|
|
|
|
2017-05-02 12:45:42 -07:00
|
|
|
When the {dfeed} is created, you receive the following results:
|
2017-04-21 11:23:27 -04:00
|
|
|
[source,js]
|
2017-04-04 15:26:39 -07:00
|
|
|
----
|
|
|
|
{
|
2017-10-30 07:54:33 -07:00
|
|
|
"datafeed_id": "datafeed-total-requests",
|
|
|
|
"job_id": "total-requests",
|
|
|
|
"query_delay": "83474ms",
|
2017-05-04 11:37:12 +01:00
|
|
|
"indices": [
|
2017-10-30 07:54:33 -07:00
|
|
|
"server-metrics"
|
2017-04-10 08:59:27 -07:00
|
|
|
],
|
|
|
|
"query": {
|
|
|
|
"match_all": {
|
2017-10-30 07:54:33 -07:00
|
|
|
"boost": 1.0
|
2017-04-10 08:59:27 -07:00
|
|
|
}
|
2017-04-04 15:26:39 -07:00
|
|
|
},
|
2017-04-11 18:52:47 -07:00
|
|
|
"scroll_size": 1000,
|
|
|
|
"chunking_config": {
|
|
|
|
"mode": "auto"
|
|
|
|
}
|
2017-04-04 15:26:39 -07:00
|
|
|
}
|
|
|
|
----
|
2018-02-09 09:16:24 -08:00
|
|
|
// TESTRESPONSE[s/"query_delay": "83474ms"/"query_delay": $body.query_delay/]
|
2018-12-07 20:34:11 +00:00
|
|
|
// TESTRESPONSE[s/"query.boost": "1.0"/"query.boost": $body.query.boost/]
|