2017-06-19 21:23:58 -04:00
|
|
|
[role="xpack"]
|
2018-08-31 19:49:24 -04:00
|
|
|
[testenv="platinum"]
|
2017-04-04 18:26:39 -04:00
|
|
|
[[ml-get-snapshot]]
|
2020-07-20 16:10:54 -04:00
|
|
|
= Get model snapshots API
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2018-12-20 13:23:28 -05:00
|
|
|
<titleabbrev>Get model snapshots</titleabbrev>
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-06-13 16:37:35 -04:00
|
|
|
Retrieves information about model snapshots.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-snapshot-request]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-request-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-12-07 15:34:11 -05:00
|
|
|
`GET _ml/anomaly_detectors/<job_id>/model_snapshots` +
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-12-07 15:34:11 -05:00
|
|
|
`GET _ml/anomaly_detectors/<job_id>/model_snapshots/<snapshot_id>`
|
2017-04-18 18:13:21 -04:00
|
|
|
|
2019-06-27 16:58:42 -04:00
|
|
|
[[ml-get-snapshot-prereqs]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-prereq-title}
|
2019-06-27 16:58:42 -04:00
|
|
|
|
|
|
|
* If the {es} {security-features} are enabled, you must have `monitor_ml`,
|
|
|
|
`monitor`, `manage_ml`, or `manage` cluster privileges to use this API. See
|
2020-07-23 19:43:10 -04:00
|
|
|
<<security-privileges>> and {ml-docs-setup-privileges}.
|
2019-06-27 16:58:42 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-snapshot-path-parms]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-path-parms-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`<job_id>`::
|
2019-12-27 16:30:26 -05:00
|
|
|
(Required, string)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`<snapshot_id>`::
|
2019-12-31 13:53:05 -05:00
|
|
|
(Optional, string)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=snapshot-id]
|
2019-12-31 13:53:05 -05:00
|
|
|
+
|
|
|
|
--
|
2020-09-18 11:06:07 -04:00
|
|
|
You can multiple snapshots for a single job in a single API request
|
|
|
|
by using a comma-separated list of `<snapshot_id>` or a wildcard expression.
|
|
|
|
You can get all snapshots for all calendars by using `_all`,
|
|
|
|
by specifying `*` as the `<snapshot_id>`, or by omitting the `<snapshot_id>`.
|
2019-12-31 13:53:05 -05:00
|
|
|
--
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-snapshot-request-body]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-request-body-title}
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`desc`::
|
|
|
|
(Optional, boolean) If true, the results are sorted in descending order.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`end`::
|
|
|
|
(Optional, date) Returns snapshots with timestamps earlier than this time.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`from`::
|
|
|
|
(Optional, integer) Skips the specified number of snapshots.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`size`::
|
|
|
|
(Optional, integer) Specifies the maximum number of snapshots to obtain.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`sort`::
|
|
|
|
(Optional, string) Specifies the sort field for the requested snapshots. By
|
|
|
|
default, the snapshots are sorted by their timestamp.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`start`::
|
|
|
|
(Optional, string) Returns snapshots with timestamps after this time.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
[role="child_attributes"]
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-snapshot-results]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-response-body-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-12-31 13:53:05 -05:00
|
|
|
The API returns an array of model snapshot objects, which have the following
|
|
|
|
properties:
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-12-31 13:53:05 -05:00
|
|
|
`description`::
|
|
|
|
(string) An optional description of the job.
|
|
|
|
|
|
|
|
`job_id`::
|
|
|
|
(string) A numerical character string that uniquely identifies the job that
|
|
|
|
the snapshot was created for.
|
|
|
|
|
|
|
|
`latest_record_time_stamp`::
|
|
|
|
(date) The timestamp of the latest processed record.
|
|
|
|
|
|
|
|
`latest_result_time_stamp`::
|
|
|
|
(date) The timestamp of the latest bucket result.
|
|
|
|
|
|
|
|
`min_version`::
|
|
|
|
(string) The minimum version required to be able to restore the model snapshot.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
//Begin model_size_stats
|
2019-12-31 13:53:05 -05:00
|
|
|
`model_size_stats`::
|
|
|
|
(object) Summary information describing the model.
|
2020-03-30 14:06:33 -04:00
|
|
|
+
|
|
|
|
.Properties of `model_size_stats`
|
|
|
|
[%collapsible%open]
|
|
|
|
====
|
|
|
|
`bucket_allocation_failures_count`:::
|
|
|
|
(long) The number of buckets for which entities were not processed due to memory
|
|
|
|
limit constraints.
|
|
|
|
|
|
|
|
`categorized_doc_count`:::
|
2020-02-10 04:10:50 -05:00
|
|
|
(long) The number of documents that have had a field categorized.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`categorization_status`:::
|
2020-02-10 04:10:50 -05:00
|
|
|
(string) The status of categorization for this job.
|
|
|
|
Contains one of the following values.
|
|
|
|
+
|
|
|
|
--
|
|
|
|
* `ok`: Categorization is performing acceptably well (or not being
|
|
|
|
used at all).
|
|
|
|
* `warn`: Categorization is detecting a distribution of categories
|
|
|
|
that suggests the input data is inappropriate for categorization.
|
|
|
|
Problems could be that there is only one category, more than 90% of
|
|
|
|
categories are rare, the number of categories is greater than 50% of
|
|
|
|
the number of categorized documents, there are no frequently
|
|
|
|
matched categories, or more than 50% of categories are dead.
|
|
|
|
|
|
|
|
--
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`dead_category_count`:::
|
2020-02-10 04:10:50 -05:00
|
|
|
(long) The number of categories created by categorization that will
|
|
|
|
never be assigned again because another category's definition
|
|
|
|
makes it a superset of the dead category. (Dead categories are a
|
|
|
|
side effect of the way categorization has no prior training.)
|
|
|
|
|
2020-04-25 05:36:49 -04:00
|
|
|
`failed_category_count`:::
|
|
|
|
(long)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=failed-category-count]
|
2020-04-25 05:36:49 -04:00
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`frequent_category_count`:::
|
2020-02-10 04:10:50 -05:00
|
|
|
(long) The number of categories that match more than 1% of categorized
|
|
|
|
documents.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`job_id`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(string)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
|
2019-12-31 13:53:05 -05:00
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`log_time`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(date) The timestamp that the `model_size_stats` were recorded, according to
|
|
|
|
server-time.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`memory_status`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(string) The status of the memory in relation to its `model_memory_limit`.
|
|
|
|
Contains one of the following values.
|
|
|
|
+
|
|
|
|
--
|
|
|
|
* `hard_limit`: The internal models require more space that the configured
|
|
|
|
memory limit. Some incoming data could not be processed.
|
|
|
|
* `ok`: The internal models stayed below the configured value.
|
|
|
|
* `soft_limit`: The internal models require more than 60% of the configured
|
|
|
|
memory limit and more aggressive pruning will be performed in order to try to
|
|
|
|
reclaim space.
|
|
|
|
--
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`model_bytes`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(long) An approximation of the memory resources required for this analysis.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`model_bytes_exceeded`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(long) The number of bytes over the high limit for memory usage at the last allocation failure.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`model_bytes_memory_limit`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(long) The upper limit for memory usage, checked on increasing values.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`rare_category_count`:::
|
2020-02-10 04:10:50 -05:00
|
|
|
(long) The number of categories that match just one categorized document.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`result_type`:::
|
|
|
|
(string) Internal. This value is always `model_size_stats`.
|
2019-12-31 13:53:05 -05:00
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`timestamp`:::
|
|
|
|
(date) The timestamp that the `model_size_stats` were recorded, according to the
|
|
|
|
bucket timestamp of the data.
|
2019-12-31 13:53:05 -05:00
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`total_by_field_count`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(long) The number of _by_ field values analyzed. Note that these are counted
|
|
|
|
separately for each detector and partition.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`total_category_count`:::
|
2020-02-10 04:10:50 -05:00
|
|
|
(long) The number of categories created by categorization.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`total_over_field_count`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(long) The number of _over_ field values analyzed. Note that these are counted
|
|
|
|
separately for each detector and partition.
|
|
|
|
|
2020-03-30 14:06:33 -04:00
|
|
|
`total_partition_field_count`:::
|
2019-12-31 13:53:05 -05:00
|
|
|
(long) The number of _partition_ field values analyzed.
|
2020-03-30 14:06:33 -04:00
|
|
|
====
|
|
|
|
//End model_size_stats
|
2019-12-31 13:53:05 -05:00
|
|
|
|
|
|
|
`retain`::
|
|
|
|
(boolean)
|
2020-06-01 16:46:15 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=retain]
|
2019-12-31 13:53:05 -05:00
|
|
|
|
|
|
|
`snapshot_id`::
|
|
|
|
(string)
|
2020-06-01 19:42:53 -04:00
|
|
|
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=snapshot-id]
|
2019-12-31 13:53:05 -05:00
|
|
|
|
|
|
|
`snapshot_doc_count`::
|
|
|
|
(long) For internal use only.
|
|
|
|
|
|
|
|
`timestamp`::
|
|
|
|
(date) The creation timestamp for the snapshot.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-get-snapshot-example]]
|
2020-07-20 16:10:54 -04:00
|
|
|
== {api-examples-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
[source,console]
|
2017-04-11 16:25:38 -04:00
|
|
|
--------------------------------------------------
|
2019-12-31 13:53:05 -05:00
|
|
|
GET _ml/anomaly_detectors/high_sum_total_sales/model_snapshots
|
2017-04-11 16:25:38 -04:00
|
|
|
{
|
2019-12-31 13:53:05 -05:00
|
|
|
"start": "1575402236000"
|
2017-04-11 16:25:38 -04:00
|
|
|
}
|
|
|
|
--------------------------------------------------
|
2019-12-31 13:53:05 -05:00
|
|
|
// TEST[skip:Kibana sample data]
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
In this example, the API provides a single result:
|
2017-04-21 11:23:27 -04:00
|
|
|
[source,js]
|
2017-04-04 18:26:39 -04:00
|
|
|
----
|
|
|
|
{
|
2019-12-31 13:53:05 -05:00
|
|
|
"count" : 1,
|
|
|
|
"model_snapshots" : [
|
2017-04-11 16:25:38 -04:00
|
|
|
{
|
2019-12-31 13:53:05 -05:00
|
|
|
"job_id" : "high_sum_total_sales",
|
|
|
|
"min_version" : "6.4.0",
|
|
|
|
"timestamp" : 1575402237000,
|
|
|
|
"description" : "State persisted due to job close at 2019-12-03T19:43:57+0000",
|
|
|
|
"snapshot_id" : "1575402237",
|
|
|
|
"snapshot_doc_count" : 1,
|
|
|
|
"model_size_stats" : {
|
|
|
|
"job_id" : "high_sum_total_sales",
|
|
|
|
"result_type" : "model_size_stats",
|
|
|
|
"model_bytes" : 1638816,
|
|
|
|
"model_bytes_exceeded" : 0,
|
|
|
|
"model_bytes_memory_limit" : 10485760,
|
|
|
|
"total_by_field_count" : 3,
|
|
|
|
"total_over_field_count" : 3320,
|
|
|
|
"total_partition_field_count" : 2,
|
|
|
|
"bucket_allocation_failures_count" : 0,
|
|
|
|
"memory_status" : "ok",
|
2020-02-10 04:10:50 -05:00
|
|
|
"categorized_doc_count" : 0,
|
|
|
|
"total_category_count" : 0,
|
|
|
|
"frequent_category_count" : 0,
|
|
|
|
"rare_category_count" : 0,
|
|
|
|
"dead_category_count" : 0,
|
|
|
|
"categorization_status" : "ok",
|
2019-12-31 13:53:05 -05:00
|
|
|
"log_time" : 1575402237000,
|
|
|
|
"timestamp" : 1576965600000
|
2017-04-04 18:26:39 -04:00
|
|
|
},
|
2019-12-31 13:53:05 -05:00
|
|
|
"latest_record_time_stamp" : 1576971072000,
|
|
|
|
"latest_result_time_stamp" : 1576965600000,
|
|
|
|
"retain" : false
|
2017-04-04 18:26:39 -04:00
|
|
|
}
|
|
|
|
]
|
|
|
|
}
|
|
|
|
----
|