[DOCS] Add property table for ML Update Jobs API (elastic/x-pack-elasticsearch#1268)
* [DOCS] Add property table for ML Update Jobs API * [DOCS] Updates based on feedback for ML Update Jobs API * [DOCS] Removed detector properties from ML Update Jobs API * [DOCS] Fixes typos Original commit: elastic/x-pack-elasticsearch@68d1b5598c
This commit is contained in:
Lisa Cawley
lcawley
parent
9b2fb6ac16
commit
33c50f1201
|
|
@ -489,9 +489,9 @@ The number of records that have been processed by the job.
|
|||
Memory status::
|
||||
The status of the mathematical models. When you create jobs by using the APIs or
|
||||
by using the advanced options in {kib}, you can specify a `model_memory_limit`.
|
||||
That value is the maximum amount of memory, in MiB, that the mathematical models
|
||||
can use. Once that limit is approached, data pruning becomes more aggressive.
|
||||
Upon exceeding that limit, new entities are not modeled.
|
||||
That value is the maximum amount of memory resources, in MiB, that the
|
||||
mathematical models can use. Once that limit is approached, data pruning becomes
|
||||
more aggressive. Upon exceeding that limit, new entities are not modeled.
|
||||
The default value is `4096`. The memory status field reflects whether you have
|
||||
reached or exceeded the model memory limit. It can have one of the following
|
||||
values: +
|
||||
|
|
|
|||
|
|
@ -260,9 +260,10 @@ The `analysis_limits` object has the following properties:
|
|||
NOTE: The `categorization_examples_limit` only applies to analysis that uses categorization.
|
||||
|
||||
`model_memory_limit`::
|
||||
(long) The maximum amount of memory, in MiB, that the mathematical models can use.
|
||||
Once this limit is approached, data pruning becomes more aggressive.
|
||||
Upon exceeding this limit, new entities are not modeled. The default value is 4096.
|
||||
(long) The approximate maximum amount of memory resources that are required
|
||||
for analytical processing, in MiB. Once this limit is approached, data pruning
|
||||
becomes more aggressive. Upon exceeding this limit, new entities are not
|
||||
modeled. The default value is 4096.
|
||||
|
||||
[float]
|
||||
[[ml-apimodelplotconfig]]
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ A model snapshot resource has the following properties:
|
|||
(string) An optional description of the job.
|
||||
|
||||
`job_id`::
|
||||
(string) A numerical character string that uniquely identifing the job that the snapshot was created for.
|
||||
(string) A numerical character string that uniquely identifies the job that the snapshot was created for.
|
||||
|
||||
`latest_record_time_stamp`::
|
||||
(date) The timestamp of the latest processed record.
|
||||
|
|
@ -30,11 +30,12 @@ A model snapshot resource has the following properties:
|
|||
(date) The timestamp of the latest bucket result.
|
||||
|
||||
`model_size_stats`::
|
||||
(object) Summary information describing the model. See <<ml-snapshot-stats,Model Size Statistics>>.
|
||||
(object) Summary information describing the model.
|
||||
See <<ml-snapshot-stats,Model Size Statistics>>.
|
||||
|
||||
`retain`::
|
||||
(boolean) If true, this snapshot will not be deleted during automatic cleanup of snapshots
|
||||
older than `model_snapshot_retention_days`.
|
||||
(boolean) If true, this snapshot will not be deleted during automatic cleanup
|
||||
of snapshots older than `model_snapshot_retention_days`.
|
||||
However, this snapshot will be deleted when the job is deleted.
|
||||
The default value is false.
|
||||
|
||||
|
|
@ -55,22 +56,25 @@ A model snapshot resource has the following properties:
|
|||
The `model_size_stats` object has the following properties:
|
||||
|
||||
`bucket_allocation_failures_count`::
|
||||
(long) The number of buckets for which entites were not processed due to memory limit constraints.
|
||||
(long) The number of buckets for which entities were not processed due to
|
||||
memory limit constraints.
|
||||
|
||||
`job_id`::
|
||||
(string) A numerical character string that uniquely identifies the job.
|
||||
|
||||
`log_time`::
|
||||
(date) The timestamp that the `model_size_stats` were recorded, according to server-time.
|
||||
(date) The timestamp that the `model_size_stats` were recorded, according to
|
||||
server-time.
|
||||
|
||||
`memory_status`::
|
||||
(string) The status of the memory in relation to its `model_memory_limit`.
|
||||
Contains one of the following values.
|
||||
`ok`::: The internal models stayed below the configured value.
|
||||
`soft_limit`::: The internal models require more than 60% of the configured memory limit and more aggressive pruning will
|
||||
`soft_limit`::: The internal models require more than 60% of the configured
|
||||
memory limit and more aggressive pruning will
|
||||
be performed in order to try to reclaim space.
|
||||
`hard_limit`::: The internal models require more space that the configured memory limit.
|
||||
Some incoming data could not be processed.
|
||||
`hard_limit`::: The internal models require more space that the configured
|
||||
memory limit. Some incoming data could not be processed.
|
||||
|
||||
`model_bytes`::
|
||||
(long) An approximation of the memory resources required for this analysis.
|
||||
|
|
@ -89,4 +93,3 @@ The `model_size_stats` object has the following properties:
|
|||
|
||||
`total_partition_field_count`::
|
||||
(long) The number of _partition_ field values analyzed.
|
||||
|
||||
|
|
|
|||
|
|
@ -9,11 +9,6 @@ The update job API enables you to update certain properties of a job.
|
|||
`POST _xpack/ml/anomaly_detectors/<job_id>/_update`
|
||||
|
||||
|
||||
////
|
||||
===== Description
|
||||
|
||||
//TBD: Important:: Updates do not take effect until after then job is closed and re-opened.
|
||||
////
|
||||
===== Path Parameters
|
||||
|
||||
`job_id` (required)::
|
||||
|
|
@ -23,16 +18,48 @@ The update job API enables you to update certain properties of a job.
|
|||
|
||||
The following properties can be updated after the job is created:
|
||||
|
||||
`analysis_config`::
|
||||
(object) The analysis configuration, which specifies how to analyze the data.
|
||||
See <<ml-analysisconfig, analysis configuration objects>>. In particular,
|
||||
the following properties can be updated: `categorization_filters`,
|
||||
`detector_description`.
|
||||
//TBD: Full list of properties that can be updated?
|
||||
[cols="<,<,<",options="header",]
|
||||
|=======================================================================
|
||||
|Name |Description |Requires Restart
|
||||
|
||||
`analysis_limits`::
|
||||
(object) Specifies runtime limits for the job.
|
||||
See <<ml-apilimits,analysis limits>>.
|
||||
|`analysis_limits`: `model_memory_limit` |The approximate maximum amount of
|
||||
memory resources required for analytical processing, in MiB.
|
||||
See <<ml-apilimits>>. | Yes
|
||||
|
||||
|`background_persist_interval` |Advanced configuration option. The time between
|
||||
each periodic persistence of the model. See <<ml-job-resource>>. | Yes
|
||||
|
||||
|`custom_settings` |Contains custom meta data about the job. | No
|
||||
|
||||
|`description` |An optional description of the job.
|
||||
See <<ml-job-resource>>. | No
|
||||
|
||||
|`model_plot_config`: `enabled` |If true, enables calculation and storage of the
|
||||
model bounds for each entity that is being analyzed.
|
||||
See <<ml-apimodelplotconfig>>. | No
|
||||
|
||||
|`model_snapshot_retention_days` |The time in days that model snapshots are
|
||||
retained for the job. See <<ml-job-resource>>. | Yes
|
||||
|
||||
|`renormalization_window_days` |Advanced configuration option. The period over
|
||||
which adjustments to the score are applied, as new data is seen.
|
||||
See <<ml-job-resource>>. | Yes
|
||||
|
||||
|`results_retention_days` |Advanced configuration option. The number of days
|
||||
for which job results are retained. See <<ml-job-resource>>. | Yes
|
||||
|
||||
|=======================================================================
|
||||
|
||||
For those properties that have `Requires Restart` set to `Yes` in this table,
|
||||
if the job is open when you make the update, you must stop the data feed, close
|
||||
the job, then restart the data feed and open the job for the changes to take
|
||||
effect.
|
||||
|
||||
//|`analysis_config`: `detectors`: `index` | A unique identifier of the
|
||||
//detector. Matches the order of detectors returned by
|
||||
//<<ml-get-job,GET job>>, starting from 0. | No
|
||||
//|`analysis_config`: `detectors`: `detector_description` |A description of the
|
||||
//detector. See <<ml-analysisconfig>>. | No
|
||||
|
||||
[NOTE]
|
||||
--
|
||||
|
|
@ -43,9 +70,6 @@ of `hard_limit`, this means that it was unable to process some data. You might
|
|||
want to re-run this job with an increased `model_memory_limit`.
|
||||
--
|
||||
|
||||
`description`::
|
||||
(string) An optional description of the job.
|
||||
|
||||
|
||||
===== Authorization
|
||||
|
||||
|
|
@ -55,32 +79,81 @@ For more information, see <<privileges-list-cluster>>.
|
|||
|
||||
===== Examples
|
||||
|
||||
The following example updates the `it-ops-kpi` job:
|
||||
The following example updates the `it_ops_new_logs` job:
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST _xpack/ml/anomaly_detectors/it-ops-kpi/_update
|
||||
POST _xpack/ml/anomaly_detectors/it_ops_new_logs/_update
|
||||
{
|
||||
"description":"New description",
|
||||
"description":"An updated job",
|
||||
"model_plot_config": {
|
||||
"enabled": true
|
||||
},
|
||||
"analysis_limits": {
|
||||
"model_memory_limit": 8192
|
||||
"model_memory_limit": 1024
|
||||
},
|
||||
"renormalization_window_days": 30,
|
||||
"background_persist_interval": "2h",
|
||||
"model_snapshot_retention_days": 7,
|
||||
"results_retention_days": 60,
|
||||
"custom_settings": {
|
||||
"custom_urls" : [{
|
||||
"url_name" : "Lookup IP",
|
||||
"url_value" : "http://geoiplookup.net/ip/$clientip$"
|
||||
}]
|
||||
}
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[skip:todo]
|
||||
|
||||
When the job is updated, you receive the following results:
|
||||
When the job is updated, you receive a summary of the job configuration
|
||||
information, including the updated property values. For example:
|
||||
|
||||
[source,js]
|
||||
----
|
||||
{
|
||||
"job_id": "it-ops-kpi",
|
||||
"job_id": "it_ops_new_logs",
|
||||
"job_type": "anomaly_detector",
|
||||
"description": "New description",
|
||||
...
|
||||
"analysis_limits": {
|
||||
"model_memory_limit": 8192
|
||||
"description": "An updated job",
|
||||
"create_time": 1493678314204,
|
||||
"finished_time": 1493678315850,
|
||||
"analysis_config": {
|
||||
"bucket_span": "1800s",
|
||||
"categorization_field_name": "message",
|
||||
"detectors": [
|
||||
{
|
||||
"detector_description": "Unusual message counts",
|
||||
"function": "count",
|
||||
"by_field_name": "mlcategory",
|
||||
"detector_rules": []
|
||||
}
|
||||
],
|
||||
"influencers": []
|
||||
},
|
||||
...
|
||||
"analysis_limits": {
|
||||
"model_memory_limit": 1024
|
||||
},
|
||||
"data_description": {
|
||||
"time_field": "time",
|
||||
"time_format": "epoch_ms"
|
||||
},
|
||||
"model_plot_config": {
|
||||
"enabled": true
|
||||
},
|
||||
"renormalization_window_days": 30,
|
||||
"background_persist_interval": "2h",
|
||||
"model_snapshot_retention_days": 7,
|
||||
"results_retention_days": 60,
|
||||
"custom_settings": {
|
||||
"custom_urls": [
|
||||
{
|
||||
"url_name": "Lookup IP",
|
||||
"url_value": "http://geoiplookup.net/ip/$clientip$"
|
||||
}
|
||||
]
|
||||
},
|
||||
"model_snapshot_id": "1493678315",
|
||||
"results_index_name": "shared"
|
||||
}
|
||||
----
|
||||
|
|
|
|||
Loading…
Reference in New Issue