2017-06-19 21:23:58 -04:00
|
|
|
[role="xpack"]
|
2018-08-31 19:49:24 -04:00
|
|
|
[testenv="platinum"]
|
2017-04-04 18:26:39 -04:00
|
|
|
[[ml-snapshot-resource]]
|
2018-12-20 13:23:28 -05:00
|
|
|
=== Model snapshot resources
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2017-04-11 16:25:38 -04:00
|
|
|
Model snapshots are saved to disk periodically.
|
2017-04-24 13:46:17 -04:00
|
|
|
By default, this is occurs approximately every 3 hours to 4 hours and is
|
|
|
|
configurable with the `background_persist_interval` property.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2018-02-21 11:58:17 -05:00
|
|
|
By default, model snapshots are retained for one day (twenty-four hours). You
|
|
|
|
can change this behavior by updating the `model_snapshot_retention_days` for the
|
|
|
|
job. When choosing a new value, consider the following:
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
* Persistence enables resilience in the event of a system failure.
|
2017-04-26 18:29:55 -04:00
|
|
|
* Persistence enables snapshots to be reverted.
|
2017-04-11 16:25:38 -04:00
|
|
|
* The time taken to persist a job is proportional to the size of the model in memory.
|
|
|
|
|
|
|
|
A model snapshot resource has the following properties:
|
|
|
|
|
|
|
|
`description`::
|
2017-04-11 22:26:18 -04:00
|
|
|
(string) An optional description of the job.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`job_id`::
|
2018-02-19 14:48:09 -05:00
|
|
|
(string) A numerical character string that uniquely identifies the job that
|
|
|
|
the snapshot was created for.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2018-03-15 13:38:52 -04:00
|
|
|
`min_version`::
|
|
|
|
(string) The minimum version required to be able to restore the model snapshot.
|
|
|
|
|
2017-04-11 16:25:38 -04:00
|
|
|
`latest_record_time_stamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The timestamp of the latest processed record.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`latest_result_time_stamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The timestamp of the latest bucket result.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`model_size_stats`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(object) Summary information describing the model.
|
|
|
|
See <<ml-snapshot-stats,Model Size Statistics>>.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`retain`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(boolean) If true, this snapshot will not be deleted during automatic cleanup
|
|
|
|
of snapshots older than `model_snapshot_retention_days`.
|
2017-04-24 12:31:31 -04:00
|
|
|
However, this snapshot will be deleted when the job is deleted.
|
|
|
|
The default value is false.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`snapshot_id`::
|
2017-04-11 22:26:18 -04:00
|
|
|
(string) A numerical character string that uniquely identifies the model
|
2017-04-11 16:25:38 -04:00
|
|
|
snapshot. For example: "1491852978".
|
|
|
|
|
|
|
|
`snapshot_doc_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) For internal use only.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`timestamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The creation timestamp for the snapshot.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2018-02-19 14:48:09 -05:00
|
|
|
NOTE: All of these properties are informational with the exception of
|
|
|
|
`description` and `retain`.
|
|
|
|
|
2017-04-11 21:52:47 -04:00
|
|
|
[float]
|
|
|
|
[[ml-snapshot-stats]]
|
2017-06-06 16:42:47 -04:00
|
|
|
==== Model Size Statistics
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
The `model_size_stats` object has the following properties:
|
|
|
|
|
|
|
|
`bucket_allocation_failures_count`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(long) The number of buckets for which entities were not processed due to
|
|
|
|
memory limit constraints.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`job_id`::
|
2017-04-11 22:26:18 -04:00
|
|
|
(string) A numerical character string that uniquely identifies the job.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`log_time`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(date) The timestamp that the `model_size_stats` were recorded, according to
|
|
|
|
server-time.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`memory_status`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(string) The status of the memory in relation to its `model_memory_limit`.
|
|
|
|
Contains one of the following values.
|
|
|
|
`ok`::: The internal models stayed below the configured value.
|
2017-05-02 18:34:30 -04:00
|
|
|
`soft_limit`::: The internal models require more than 60% of the configured
|
|
|
|
memory limit and more aggressive pruning will
|
2017-04-24 12:31:31 -04:00
|
|
|
be performed in order to try to reclaim space.
|
2017-05-02 18:34:30 -04:00
|
|
|
`hard_limit`::: The internal models require more space that the configured
|
|
|
|
memory limit. Some incoming data could not be processed.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`model_bytes`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) An approximation of the memory resources required for this analysis.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`result_type`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(string) Internal. This value is always set to "model_size_stats".
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`timestamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The timestamp that the `model_size_stats` were recorded, according to the bucket timestamp of the data.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`total_by_field_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) The number of _by_ field values analyzed. Note that these are counted separately for each detector and partition.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`total_over_field_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) The number of _over_ field values analyzed. Note that these are counted separately for each detector and partition.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`total_partition_field_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) The number of _partition_ field values analyzed.
|
2018-02-19 14:48:09 -05:00
|
|
|
|
|
|
|
NOTE: All of these properties are informational; you cannot change their values.
|