2017-04-11 21:52:47 -04:00
|
|
|
//lcawley Verified example output 2017-04-11
|
2017-04-04 18:26:39 -04:00
|
|
|
[[ml-snapshot-resource]]
|
|
|
|
==== Model Snapshot Resources
|
|
|
|
|
2017-04-11 16:25:38 -04:00
|
|
|
Model snapshots are saved to disk periodically.
|
2017-04-24 13:46:17 -04:00
|
|
|
By default, this is occurs approximately every 3 hours to 4 hours and is
|
|
|
|
configurable with the `background_persist_interval` property.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
By default, model snapshots are retained for one day. You can change this
|
2017-04-24 12:31:31 -04:00
|
|
|
behavior by updating the `model_snapshot_retention_days` for the job.
|
2017-04-11 16:25:38 -04:00
|
|
|
When choosing a new value, consider the following:
|
|
|
|
|
|
|
|
* Persistence enables resilience in the event of a system failure.
|
2017-04-26 18:29:55 -04:00
|
|
|
* Persistence enables snapshots to be reverted.
|
2017-04-11 16:25:38 -04:00
|
|
|
* The time taken to persist a job is proportional to the size of the model in memory.
|
|
|
|
//* The smallest allowed value is 3600 (1 hour).
|
|
|
|
|
|
|
|
A model snapshot resource has the following properties:
|
|
|
|
|
|
|
|
`description`::
|
2017-04-11 22:26:18 -04:00
|
|
|
(string) An optional description of the job.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`job_id`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(string) A numerical character string that uniquely identifies the job that the snapshot was created for.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`latest_record_time_stamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The timestamp of the latest processed record.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`latest_result_time_stamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The timestamp of the latest bucket result.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`model_size_stats`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(object) Summary information describing the model.
|
|
|
|
See <<ml-snapshot-stats,Model Size Statistics>>.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`retain`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(boolean) If true, this snapshot will not be deleted during automatic cleanup
|
|
|
|
of snapshots older than `model_snapshot_retention_days`.
|
2017-04-24 12:31:31 -04:00
|
|
|
However, this snapshot will be deleted when the job is deleted.
|
|
|
|
The default value is false.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`snapshot_id`::
|
2017-04-11 22:26:18 -04:00
|
|
|
(string) A numerical character string that uniquely identifies the model
|
2017-04-11 16:25:38 -04:00
|
|
|
snapshot. For example: "1491852978".
|
|
|
|
|
|
|
|
`snapshot_doc_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) For internal use only.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`timestamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The creation timestamp for the snapshot.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
2017-04-11 21:52:47 -04:00
|
|
|
[float]
|
|
|
|
[[ml-snapshot-stats]]
|
2017-04-11 16:25:38 -04:00
|
|
|
===== Model Size Statistics
|
|
|
|
|
|
|
|
The `model_size_stats` object has the following properties:
|
|
|
|
|
|
|
|
`bucket_allocation_failures_count`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(long) The number of buckets for which entities were not processed due to
|
|
|
|
memory limit constraints.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`job_id`::
|
2017-04-11 22:26:18 -04:00
|
|
|
(string) A numerical character string that uniquely identifies the job.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`log_time`::
|
2017-05-02 18:34:30 -04:00
|
|
|
(date) The timestamp that the `model_size_stats` were recorded, according to
|
|
|
|
server-time.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`memory_status`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(string) The status of the memory in relation to its `model_memory_limit`.
|
|
|
|
Contains one of the following values.
|
|
|
|
`ok`::: The internal models stayed below the configured value.
|
2017-05-02 18:34:30 -04:00
|
|
|
`soft_limit`::: The internal models require more than 60% of the configured
|
|
|
|
memory limit and more aggressive pruning will
|
2017-04-24 12:31:31 -04:00
|
|
|
be performed in order to try to reclaim space.
|
2017-05-02 18:34:30 -04:00
|
|
|
`hard_limit`::: The internal models require more space that the configured
|
|
|
|
memory limit. Some incoming data could not be processed.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`model_bytes`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) An approximation of the memory resources required for this analysis.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`result_type`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(string) Internal. This value is always set to "model_size_stats".
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`timestamp`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(date) The timestamp that the `model_size_stats` were recorded, according to the bucket timestamp of the data.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`total_by_field_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) The number of _by_ field values analyzed. Note that these are counted separately for each detector and partition.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`total_over_field_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) The number of _over_ field values analyzed. Note that these are counted separately for each detector and partition.
|
2017-04-11 16:25:38 -04:00
|
|
|
|
|
|
|
`total_partition_field_count`::
|
2017-04-24 12:31:31 -04:00
|
|
|
(long) The number of _partition_ field values analyzed.
|