2017-06-19 21:23:58 -04:00
|
|
|
[role="xpack"]
|
2018-08-31 19:49:24 -04:00
|
|
|
[testenv="platinum"]
|
2017-04-04 18:26:39 -04:00
|
|
|
[[ml-get-category]]
|
2020-07-20 16:10:54 -04:00
|
|
|
= Get categories API
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2018-12-20 13:23:28 -05:00
|
|
|
<titleabbrev>Get categories</titleabbrev>
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-07-30 13:52:23 -04:00
|
|
|
Retrieves {anomaly-job} results for one or more categories.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-category-request]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-request-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-12-07 15:34:11 -05:00
|
|
|
`GET _ml/anomaly_detectors/<job_id>/results/categories` +
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-12-07 15:34:11 -05:00
|
|
|
`GET _ml/anomaly_detectors/<job_id>/results/categories/<category_id>`
|
2017-04-18 18:13:21 -04:00
|
|
|
|
2019-06-27 16:58:42 -04:00
|
|
|
[[ml-get-category-prereqs]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-prereq-title}
|
2019-06-27 16:58:42 -04:00
|
|
|
|
|
|
|
* If the {es} {security-features} are enabled, you must have `monitor_ml`,
|
|
|
|
`monitor`, `manage_ml`, or `manage` cluster privileges to use this API. You also
|
|
|
|
need `read` index privilege on the index that stores the results. The
|
|
|
|
`machine_learning_admin` and `machine_learning_user` roles provide these
|
2019-10-07 18:23:19 -04:00
|
|
|
privileges. See <<security-privileges>> and
|
|
|
|
<<built-in-roles>>.
|
2019-06-27 16:58:42 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-category-desc]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-description-title}
|
2017-06-12 13:41:14 -04:00
|
|
|
|
2019-12-31 16:21:17 -05:00
|
|
|
When `categorization_field_name` is specified in the job configuration, it is
|
|
|
|
possible to view the definitions of the resulting categories. A category
|
|
|
|
definition describes the common terms matched and contains examples of matched
|
|
|
|
values.
|
|
|
|
|
|
|
|
The anomaly results from a categorization analysis are available as bucket,
|
|
|
|
influencer, and record results. For example, the results might indicate that
|
|
|
|
at 16:45 there was an unusual count of log message category 11. You can then
|
|
|
|
examine the description and examples of that category. For more information, see
|
2019-12-20 13:01:19 -05:00
|
|
|
{ml-docs}/ml-configuring-categories.html[Categorizing log messages].
|
2017-04-18 18:13:21 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-category-path-parms]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-path-parms-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-12-31 16:21:17 -05:00
|
|
|
`<category_id>`::
|
2020-06-06 03:15:17 -04:00
|
|
|
(Optional, long) Identifier for the category, which is unique in the job. If you
|
|
|
|
specify neither the category ID nor the `partition_field_value`, the API returns
|
|
|
|
information about all categories. If you specify only the
|
|
|
|
`partition_field_value`, it returns information about all categories for the
|
|
|
|
specified partition.
|
2019-12-31 16:21:17 -05:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`<job_id>`::
|
2019-12-27 16:30:26 -05:00
|
|
|
(Required, string)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-category-request-body]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-request-body-title}
|
2017-04-10 19:14:26 -04:00
|
|
|
|
2019-12-31 16:21:17 -05:00
|
|
|
`page`.`from`::
|
|
|
|
(Optional, integer) Skips the specified number of categories.
|
|
|
|
|
|
|
|
`page`.`size`::
|
|
|
|
(Optional, integer) Specifies the maximum number of categories to obtain.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2020-06-06 03:15:17 -04:00
|
|
|
`partition_field_value`::
|
|
|
|
(Optional, string) Only return categories for the specified partition.
|
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-category-results]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-response-body-title}
|
2017-04-10 19:14:26 -04:00
|
|
|
|
2019-12-31 16:21:17 -05:00
|
|
|
The API returns an array of category objects, which have the following
|
|
|
|
properties:
|
|
|
|
|
|
|
|
`category_id`::
|
2020-06-06 03:15:17 -04:00
|
|
|
(unsigned integer) A unique identifier for the category. `category_id` is unique
|
|
|
|
at the job level, even when per-partition categorization is enabled.
|
|
|
|
|
2019-12-31 16:21:17 -05:00
|
|
|
|
|
|
|
`examples`::
|
|
|
|
(array) A list of examples of actual values that matched the category.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-12-31 16:21:17 -05:00
|
|
|
`grok_pattern`::
|
|
|
|
experimental[] (string) A Grok pattern that could be used in {ls} or an ingest
|
|
|
|
pipeline to extract fields from messages that match the category. This field is
|
|
|
|
experimental and may be changed or removed in a future release. The Grok
|
|
|
|
patterns that are found are not optimal, but are often a good starting point for
|
|
|
|
manual tweaking.
|
|
|
|
|
|
|
|
`job_id`::
|
|
|
|
(string)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
|
2019-12-31 16:21:17 -05:00
|
|
|
|
|
|
|
`max_matching_length`::
|
|
|
|
(unsigned integer) The maximum length of the fields that matched the category.
|
|
|
|
The value is increased by 10% to enable matching for similar fields that have
|
|
|
|
not been analyzed.
|
|
|
|
|
2020-06-06 03:15:17 -04:00
|
|
|
// This doesn't use the shared description because there are
|
|
|
|
// categorization-specific aspects to its use in this context
|
|
|
|
`partition_field_name`::
|
|
|
|
(string) If per-partition categorization is enabled, this property identifies
|
|
|
|
the field used to segment the categorization. It is not present when
|
|
|
|
per-partition categorization is disabled.
|
|
|
|
|
|
|
|
`partition_field_value`::
|
|
|
|
(string) If per-partition categorization is enabled, this property identifies
|
|
|
|
the value of the `partition_field_name` for the category. It is not present when
|
|
|
|
per-partition categorization is disabled.
|
|
|
|
|
2019-12-31 16:21:17 -05:00
|
|
|
`regex`::
|
|
|
|
(string) A regular expression that is used to search for values that match the
|
|
|
|
category.
|
|
|
|
|
|
|
|
`terms`::
|
|
|
|
(string) A space separated list of the common tokens that are matched in values
|
|
|
|
of the category.
|
2017-04-25 17:08:29 -04:00
|
|
|
|
2020-04-02 09:09:19 -04:00
|
|
|
`num_matches`::
|
|
|
|
(long) The number of messages that have been matched by this category. This is
|
|
|
|
only guaranteed to have the latest accurate count after a job `_flush` or `_close`
|
|
|
|
|
|
|
|
`preferred_to_categories`::
|
|
|
|
(list) A list of `category_id` entries that this current category encompasses.
|
|
|
|
Any new message that is processed by the categorizer will match against this
|
|
|
|
category and not any of the categories in this list. This is only guaranteed
|
|
|
|
to have the latest accurate list of categories after a job `_flush` or `_close`
|
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-category-example]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-examples-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
[source,console]
|
2017-04-10 19:14:26 -04:00
|
|
|
--------------------------------------------------
|
2018-12-07 15:34:11 -05:00
|
|
|
GET _ml/anomaly_detectors/esxi_log/results/categories
|
2017-04-28 12:14:53 -04:00
|
|
|
{
|
|
|
|
"page":{
|
|
|
|
"size": 1
|
|
|
|
}
|
|
|
|
}
|
2017-04-10 19:14:26 -04:00
|
|
|
--------------------------------------------------
|
|
|
|
// TEST[skip:todo]
|
|
|
|
|
2017-04-28 12:14:53 -04:00
|
|
|
In this example, the API returns the following information:
|
2017-04-21 11:23:27 -04:00
|
|
|
[source,js]
|
2017-04-04 18:26:39 -04:00
|
|
|
----
|
|
|
|
{
|
2017-04-10 19:14:26 -04:00
|
|
|
"count": 11,
|
|
|
|
"categories": [
|
|
|
|
{
|
2018-05-15 04:02:38 -04:00
|
|
|
"job_id" : "esxi_log",
|
|
|
|
"category_id" : 1,
|
|
|
|
"terms" : "Vpxa verbose vpxavpxaInvtVm opID VpxaInvtVmChangeListener Guest DiskInfo Changed",
|
|
|
|
"regex" : ".*?Vpxa.+?verbose.+?vpxavpxaInvtVm.+?opID.+?VpxaInvtVmChangeListener.+?Guest.+?DiskInfo.+?Changed.*",
|
|
|
|
"max_matching_length": 154,
|
|
|
|
"examples" : [
|
|
|
|
"Oct 19 17:04:44 esxi1.acme.com Vpxa: [3CB3FB90 verbose 'vpxavpxaInvtVm' opID=WFU-33d82c31] [VpxaInvtVmChangeListener] Guest DiskInfo Changed",
|
|
|
|
"Oct 19 17:04:45 esxi2.acme.com Vpxa: [3CA66B90 verbose 'vpxavpxaInvtVm' opID=WFU-33927856] [VpxaInvtVmChangeListener] Guest DiskInfo Changed",
|
|
|
|
"Oct 19 17:04:51 esxi1.acme.com Vpxa: [FFDBAB90 verbose 'vpxavpxaInvtVm' opID=WFU-25e0d447] [VpxaInvtVmChangeListener] Guest DiskInfo Changed",
|
|
|
|
"Oct 19 17:04:58 esxi2.acme.com Vpxa: [FFDDBB90 verbose 'vpxavpxaInvtVm' opID=WFU-bbff0134] [VpxaInvtVmChangeListener] Guest DiskInfo Changed"
|
|
|
|
],
|
|
|
|
"grok_pattern" : ".*?%{SYSLOGTIMESTAMP:timestamp}.+?Vpxa.+?%{BASE16NUM:field}.+?verbose.+?vpxavpxaInvtVm.+?opID.+?VpxaInvtVmChangeListener.+?Guest.+?DiskInfo.+?Changed.*"
|
2017-04-28 12:14:53 -04:00
|
|
|
}
|
2017-04-04 18:26:39 -04:00
|
|
|
]
|
|
|
|
}
|
|
|
|
----
|