2017-04-04 18:26:39 -04:00
|
|
|
|
[[ml-introduction]]
|
|
|
|
|
== Introduction
|
|
|
|
|
|
|
|
|
|
Machine learning in {xpack} automates the analysis of time-series data by
|
|
|
|
|
creating accurate baselines of normal behaviors in the data, and identifying
|
|
|
|
|
anomalous patterns in that data.
|
|
|
|
|
|
2017-04-10 11:59:27 -04:00
|
|
|
|
Driven by proprietary machine learning algorithms, anomalies related to
|
|
|
|
|
temporal deviations in values/counts/frequencies, statistical rarity, and unusual
|
2017-04-04 18:26:39 -04:00
|
|
|
|
behaviors for a member of a population are detected, scored and linked with
|
|
|
|
|
statistically significant influencers in the data.
|
|
|
|
|
|
|
|
|
|
Automated periodicity detection and quick adaptation to changing data ensure
|
|
|
|
|
that you don’t need to specify algorithms, models, or other data
|
|
|
|
|
science-related configurations in order to get the benefits of {ml}.
|
2017-04-17 22:58:19 -04:00
|
|
|
|
//image::images/graph-network.jpg["Graph network"]
|
2017-04-04 18:26:39 -04:00
|
|
|
|
|
2017-04-10 11:59:27 -04:00
|
|
|
|
[float]
|
2017-04-04 18:26:39 -04:00
|
|
|
|
=== Integration with the Elastic Stack
|
|
|
|
|
|
|
|
|
|
Machine learning is tightly integrated with the Elastic Stack.
|
|
|
|
|
Data is pulled from {es} for analysis and anomaly results are displayed in
|
|
|
|
|
{kb} dashboards.
|
|
|
|
|
|
2017-04-10 11:59:27 -04:00
|
|
|
|
[float]
|
|
|
|
|
[[ml-concepts]]
|
|
|
|
|
=== Basic Concepts
|
|
|
|
|
|
|
|
|
|
There are a few concepts that are core to {ml} in {xpack}.
|
|
|
|
|
Understanding these concepts from the outset will tremendously help ease the
|
|
|
|
|
learning process.
|
|
|
|
|
|
|
|
|
|
Jobs::
|
|
|
|
|
Machine learning jobs contain the configuration information and metadata
|
|
|
|
|
necessary to perform an analytics task. For a list of the properties associated
|
|
|
|
|
with a job, see <<ml-job-resource, Job Resources>>.
|
|
|
|
|
|
|
|
|
|
Data feeds::
|
|
|
|
|
Jobs can analyze either a batch of data from a data store or a stream of data
|
|
|
|
|
in real-time. The latter involves data that is retrieved from {es} and is
|
2017-04-17 14:53:31 -04:00
|
|
|
|
referred to as a data feed.
|
2017-04-10 11:59:27 -04:00
|
|
|
|
|
|
|
|
|
Detectors::
|
|
|
|
|
Part of the configuration information associated with a job, detectors define
|
|
|
|
|
the type of analysis that needs to be done (for example, max, average, rare).
|
|
|
|
|
They also specify which fields to analyze. You can have more than one detector
|
|
|
|
|
in a job, which is more efficient than running multiple jobs against the same
|
|
|
|
|
data stream. For a list of the properties associated with detectors, see
|
|
|
|
|
<<ml-detectorconfig, Detector Configuration Objects>>.
|
|
|
|
|
|
|
|
|
|
Buckets::
|
|
|
|
|
Part of the configuration information associated with a job, the _bucket span_
|
|
|
|
|
defines the time interval across which the job analyzes. When setting the
|
|
|
|
|
bucket span, take into account the granularity at which you want to analyze,
|
|
|
|
|
the frequency of the input data, and the frequency at which alerting is required.
|
|
|
|
|
|
2017-04-17 14:53:31 -04:00
|
|
|
|
Machine learning nodes::
|
|
|
|
|
A {ml} node is a node that has `xpack.ml.enabled` and `node.ml` set to `true`,
|
|
|
|
|
which is the default behavior. If you set `node.ml` to `false`, the node can
|
|
|
|
|
service API requests but it cannot run jobs. If you want to use {xpack} {ml}
|
|
|
|
|
features, there must be at least one {ml} node in your cluster.
|
|
|
|
|
For more information about this setting, see <<ml-settings>>.
|
2017-04-10 11:59:27 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-04-04 18:26:39 -04:00
|
|
|
|
//[float]
|
|
|
|
|
//== Where to Go Next
|
|
|
|
|
|
|
|
|
|
//<<ml-getting-started, Getting Started>> :: Enable machine learning and start
|
|
|
|
|
//discovering anomalies in your data.
|
|
|
|
|
|
|
|
|
|
//[float]
|
|
|
|
|
//== Have Comments, Questions, or Feedback?
|
|
|
|
|
|
|
|
|
|
//Head over to our {forum}[Graph Discussion Forum] to share your experience, questions, and
|
|
|
|
|
//suggestions.
|