2017-06-19 21:01:52 -04:00
|
|
|
[role="xpack"]
|
2017-04-17 14:53:31 -04:00
|
|
|
[[ml-settings]]
|
2018-09-28 12:41:14 -04:00
|
|
|
=== Machine learning settings in Elasticsearch
|
2017-08-11 13:00:35 -04:00
|
|
|
++++
|
2018-09-28 12:41:14 -04:00
|
|
|
<titleabbrev>Machine learning settings</titleabbrev>
|
2017-08-11 13:00:35 -04:00
|
|
|
++++
|
|
|
|
|
2017-04-27 13:51:48 -04:00
|
|
|
You do not need to configure any settings to use {ml}. It is enabled by default.
|
2017-04-17 14:53:31 -04:00
|
|
|
|
2018-09-28 12:41:14 -04:00
|
|
|
All of these settings can be added to the `elasticsearch.yml` configuration file.
|
|
|
|
The dynamic settings can also be updated across a cluster with the
|
|
|
|
<<cluster-update-settings,cluster update settings API>>.
|
|
|
|
|
|
|
|
TIP: Dynamic settings take precedence over settings in the `elasticsearch.yml`
|
|
|
|
file.
|
|
|
|
|
2017-04-17 14:53:31 -04:00
|
|
|
[float]
|
|
|
|
[[general-ml-settings]]
|
2018-09-28 12:41:14 -04:00
|
|
|
==== General machine learning settings
|
2017-04-17 14:53:31 -04:00
|
|
|
|
2017-10-25 12:00:53 -04:00
|
|
|
`node.ml`::
|
|
|
|
Set to `true` (default) to identify the node as a _machine learning node_. +
|
|
|
|
+
|
|
|
|
If set to `false` in `elasticsearch.yml`, the node cannot run jobs. If set to
|
|
|
|
`true` but `xpack.ml.enabled` is set to `false`, the `node.ml` setting is
|
|
|
|
ignored and the node cannot run jobs. If you want to run jobs, there must be at
|
|
|
|
least one machine learning node in your cluster. +
|
|
|
|
+
|
|
|
|
IMPORTANT: On dedicated coordinating nodes or dedicated master nodes, disable
|
|
|
|
the `node.ml` role.
|
|
|
|
|
2017-04-17 14:53:31 -04:00
|
|
|
`xpack.ml.enabled`::
|
2017-10-13 04:22:21 -04:00
|
|
|
Set to `true` (default) to enable {ml} on the node. +
|
2017-04-17 14:53:31 -04:00
|
|
|
+
|
2017-10-13 04:22:21 -04:00
|
|
|
If set to `false` in `elasticsearch.yml`, the {ml} APIs are disabled on the node.
|
|
|
|
Therefore the node cannot open jobs, start {dfeeds}, or receive transport (internal)
|
2017-08-04 11:59:50 -04:00
|
|
|
communication requests related to {ml} APIs. It also affects all {kib} instances
|
|
|
|
that connect to this {es} instance; you do not need to disable {ml} in those
|
|
|
|
`kibana.yml` files. For more information about disabling {ml} in specific {kib}
|
|
|
|
instances, see
|
|
|
|
{kibana-ref}/ml-settings-kb.html[{kib} Machine Learning Settings].
|
2017-04-17 14:53:31 -04:00
|
|
|
+
|
2017-06-05 16:02:12 -04:00
|
|
|
IMPORTANT: If you want to use {ml} features in your cluster, you must have
|
|
|
|
`xpack.ml.enabled` set to `true` on all master-eligible nodes. This is the
|
|
|
|
default behavior.
|
2017-04-17 14:53:31 -04:00
|
|
|
|
2017-11-21 04:51:52 -05:00
|
|
|
`xpack.ml.max_machine_memory_percent`::
|
|
|
|
The maximum percentage of the machine's memory that {ml} may use for running
|
|
|
|
analytics processes. (These processes are separate to the {es} JVM.) Defaults to
|
|
|
|
`30` percent. The limit is based on the total memory of the machine, not current
|
|
|
|
free memory. Jobs will not be allocated to a node if doing so would cause the
|
|
|
|
estimated memory use of {ml} jobs to exceed the limit.
|
|
|
|
|
2017-10-25 12:00:53 -04:00
|
|
|
`xpack.ml.max_model_memory_limit`::
|
|
|
|
The maximum `model_memory_limit` property value that can be set for any job on
|
|
|
|
this node. If you try to create a job with a `model_memory_limit` property value
|
|
|
|
that is greater than this setting value, an error occurs. Existing jobs are not
|
|
|
|
affected when you update this setting. For more information about the
|
|
|
|
`model_memory_limit` property, see <<ml-apilimits>>.
|
2017-12-15 14:19:11 -05:00
|
|
|
|
2018-09-28 12:41:14 -04:00
|
|
|
`xpack.ml.max_open_jobs`::
|
|
|
|
The maximum number of jobs that can run on a node. Defaults to `20`.
|
|
|
|
The maximum number of jobs is also constrained by memory usage, so fewer
|
|
|
|
jobs than specified by this setting will run on a node if the estimated
|
|
|
|
memory use of the jobs would be higher than allowed.
|
|
|
|
|
2017-12-15 14:19:11 -05:00
|
|
|
`xpack.ml.node_concurrent_job_allocations`::
|
|
|
|
The maximum number of jobs that can concurrently be in the `opening` state on
|
|
|
|
each node. Typically, jobs spend a small amount of time in this state before
|
|
|
|
they move to `open` state. Jobs that must restore large models when they are
|
|
|
|
opening spend more time in the `opening` state. Defaults to `2`.
|
2018-09-28 12:41:14 -04:00
|
|
|
|
|
|
|
[float]
|
|
|
|
[[advanced-ml-settings]]
|
|
|
|
==== Advanced machine learning settings
|
|
|
|
|
|
|
|
These settings are for advanced use cases; the default values are generally
|
|
|
|
sufficient:
|
|
|
|
|
|
|
|
`xpack.ml.max_anomaly_records`:: (<<cluster-update-settings,Dynamic>>)
|
|
|
|
The maximum number of records that are output per bucket. The default value is
|
|
|
|
`500`.
|
|
|
|
|
2018-10-18 17:11:36 -04:00
|
|
|
`xpack.ml.max_lazy_ml_nodes`:: (<<cluster-update-settings,Dynamic>>)
|
|
|
|
The number of lazily spun up Machine Learning nodes. Useful in situations
|
|
|
|
where ML nodes are not desired until the first Machine Learning Job
|
|
|
|
is opened. It defaults to `0` and has a maximum acceptable value of `3`.
|
|
|
|
If the current number of ML nodes is `>=` than this setting, then it is
|
|
|
|
assumed that there are no more lazy nodes available as the desired number
|
|
|
|
of nodes have already been provisioned. When a job is opened with this
|
|
|
|
setting set at `>0` and there are no nodes that can accept the job, then
|
|
|
|
the job will stay in the `OPENING` state until a new ML node is added to the
|
|
|
|
cluster and the job is assigned to run on that node.
|
|
|
|
+
|
|
|
|
IMPORTANT: This setting assumes some external process is capable of adding ML nodes
|
|
|
|
to the cluster. This setting is only useful when used in conjunction with
|
|
|
|
such an external process.
|