[DOCS] Add property table for ML Update Jobs API (elastic/x-pack-elasticsearch#1268)
* [DOCS] Add property table for ML Update Jobs API * [DOCS] Updates based on feedback for ML Update Jobs API * [DOCS] Removed detector properties from ML Update Jobs API * [DOCS] Fixes typos Original commit: elastic/x-pack-elasticsearch@68d1b5598c
This commit is contained in:
parent
9b2fb6ac16
commit
33c50f1201
|
@ -489,9 +489,9 @@ The number of records that have been processed by the job.
|
||||||
Memory status::
|
Memory status::
|
||||||
The status of the mathematical models. When you create jobs by using the APIs or
|
The status of the mathematical models. When you create jobs by using the APIs or
|
||||||
by using the advanced options in {kib}, you can specify a `model_memory_limit`.
|
by using the advanced options in {kib}, you can specify a `model_memory_limit`.
|
||||||
That value is the maximum amount of memory, in MiB, that the mathematical models
|
That value is the maximum amount of memory resources, in MiB, that the
|
||||||
can use. Once that limit is approached, data pruning becomes more aggressive.
|
mathematical models can use. Once that limit is approached, data pruning becomes
|
||||||
Upon exceeding that limit, new entities are not modeled.
|
more aggressive. Upon exceeding that limit, new entities are not modeled.
|
||||||
The default value is `4096`. The memory status field reflects whether you have
|
The default value is `4096`. The memory status field reflects whether you have
|
||||||
reached or exceeded the model memory limit. It can have one of the following
|
reached or exceeded the model memory limit. It can have one of the following
|
||||||
values: +
|
values: +
|
||||||
|
|
|
@ -260,9 +260,10 @@ The `analysis_limits` object has the following properties:
|
||||||
NOTE: The `categorization_examples_limit` only applies to analysis that uses categorization.
|
NOTE: The `categorization_examples_limit` only applies to analysis that uses categorization.
|
||||||
|
|
||||||
`model_memory_limit`::
|
`model_memory_limit`::
|
||||||
(long) The maximum amount of memory, in MiB, that the mathematical models can use.
|
(long) The approximate maximum amount of memory resources that are required
|
||||||
Once this limit is approached, data pruning becomes more aggressive.
|
for analytical processing, in MiB. Once this limit is approached, data pruning
|
||||||
Upon exceeding this limit, new entities are not modeled. The default value is 4096.
|
becomes more aggressive. Upon exceeding this limit, new entities are not
|
||||||
|
modeled. The default value is 4096.
|
||||||
|
|
||||||
[float]
|
[float]
|
||||||
[[ml-apimodelplotconfig]]
|
[[ml-apimodelplotconfig]]
|
||||||
|
|
|
@ -21,7 +21,7 @@ A model snapshot resource has the following properties:
|
||||||
(string) An optional description of the job.
|
(string) An optional description of the job.
|
||||||
|
|
||||||
`job_id`::
|
`job_id`::
|
||||||
(string) A numerical character string that uniquely identifing the job that the snapshot was created for.
|
(string) A numerical character string that uniquely identifies the job that the snapshot was created for.
|
||||||
|
|
||||||
`latest_record_time_stamp`::
|
`latest_record_time_stamp`::
|
||||||
(date) The timestamp of the latest processed record.
|
(date) The timestamp of the latest processed record.
|
||||||
|
@ -30,11 +30,12 @@ A model snapshot resource has the following properties:
|
||||||
(date) The timestamp of the latest bucket result.
|
(date) The timestamp of the latest bucket result.
|
||||||
|
|
||||||
`model_size_stats`::
|
`model_size_stats`::
|
||||||
(object) Summary information describing the model. See <<ml-snapshot-stats,Model Size Statistics>>.
|
(object) Summary information describing the model.
|
||||||
|
See <<ml-snapshot-stats,Model Size Statistics>>.
|
||||||
|
|
||||||
`retain`::
|
`retain`::
|
||||||
(boolean) If true, this snapshot will not be deleted during automatic cleanup of snapshots
|
(boolean) If true, this snapshot will not be deleted during automatic cleanup
|
||||||
older than `model_snapshot_retention_days`.
|
of snapshots older than `model_snapshot_retention_days`.
|
||||||
However, this snapshot will be deleted when the job is deleted.
|
However, this snapshot will be deleted when the job is deleted.
|
||||||
The default value is false.
|
The default value is false.
|
||||||
|
|
||||||
|
@ -55,22 +56,25 @@ A model snapshot resource has the following properties:
|
||||||
The `model_size_stats` object has the following properties:
|
The `model_size_stats` object has the following properties:
|
||||||
|
|
||||||
`bucket_allocation_failures_count`::
|
`bucket_allocation_failures_count`::
|
||||||
(long) The number of buckets for which entites were not processed due to memory limit constraints.
|
(long) The number of buckets for which entities were not processed due to
|
||||||
|
memory limit constraints.
|
||||||
|
|
||||||
`job_id`::
|
`job_id`::
|
||||||
(string) A numerical character string that uniquely identifies the job.
|
(string) A numerical character string that uniquely identifies the job.
|
||||||
|
|
||||||
`log_time`::
|
`log_time`::
|
||||||
(date) The timestamp that the `model_size_stats` were recorded, according to server-time.
|
(date) The timestamp that the `model_size_stats` were recorded, according to
|
||||||
|
server-time.
|
||||||
|
|
||||||
`memory_status`::
|
`memory_status`::
|
||||||
(string) The status of the memory in relation to its `model_memory_limit`.
|
(string) The status of the memory in relation to its `model_memory_limit`.
|
||||||
Contains one of the following values.
|
Contains one of the following values.
|
||||||
`ok`::: The internal models stayed below the configured value.
|
`ok`::: The internal models stayed below the configured value.
|
||||||
`soft_limit`::: The internal models require more than 60% of the configured memory limit and more aggressive pruning will
|
`soft_limit`::: The internal models require more than 60% of the configured
|
||||||
|
memory limit and more aggressive pruning will
|
||||||
be performed in order to try to reclaim space.
|
be performed in order to try to reclaim space.
|
||||||
`hard_limit`::: The internal models require more space that the configured memory limit.
|
`hard_limit`::: The internal models require more space that the configured
|
||||||
Some incoming data could not be processed.
|
memory limit. Some incoming data could not be processed.
|
||||||
|
|
||||||
`model_bytes`::
|
`model_bytes`::
|
||||||
(long) An approximation of the memory resources required for this analysis.
|
(long) An approximation of the memory resources required for this analysis.
|
||||||
|
@ -89,4 +93,3 @@ The `model_size_stats` object has the following properties:
|
||||||
|
|
||||||
`total_partition_field_count`::
|
`total_partition_field_count`::
|
||||||
(long) The number of _partition_ field values analyzed.
|
(long) The number of _partition_ field values analyzed.
|
||||||
|
|
||||||
|
|
|
@ -9,11 +9,6 @@ The update job API enables you to update certain properties of a job.
|
||||||
`POST _xpack/ml/anomaly_detectors/<job_id>/_update`
|
`POST _xpack/ml/anomaly_detectors/<job_id>/_update`
|
||||||
|
|
||||||
|
|
||||||
////
|
|
||||||
===== Description
|
|
||||||
|
|
||||||
//TBD: Important:: Updates do not take effect until after then job is closed and re-opened.
|
|
||||||
////
|
|
||||||
===== Path Parameters
|
===== Path Parameters
|
||||||
|
|
||||||
`job_id` (required)::
|
`job_id` (required)::
|
||||||
|
@ -23,16 +18,48 @@ The update job API enables you to update certain properties of a job.
|
||||||
|
|
||||||
The following properties can be updated after the job is created:
|
The following properties can be updated after the job is created:
|
||||||
|
|
||||||
`analysis_config`::
|
[cols="<,<,<",options="header",]
|
||||||
(object) The analysis configuration, which specifies how to analyze the data.
|
|=======================================================================
|
||||||
See <<ml-analysisconfig, analysis configuration objects>>. In particular,
|
|Name |Description |Requires Restart
|
||||||
the following properties can be updated: `categorization_filters`,
|
|
||||||
`detector_description`.
|
|
||||||
//TBD: Full list of properties that can be updated?
|
|
||||||
|
|
||||||
`analysis_limits`::
|
|`analysis_limits`: `model_memory_limit` |The approximate maximum amount of
|
||||||
(object) Specifies runtime limits for the job.
|
memory resources required for analytical processing, in MiB.
|
||||||
See <<ml-apilimits,analysis limits>>.
|
See <<ml-apilimits>>. | Yes
|
||||||
|
|
||||||
|
|`background_persist_interval` |Advanced configuration option. The time between
|
||||||
|
each periodic persistence of the model. See <<ml-job-resource>>. | Yes
|
||||||
|
|
||||||
|
|`custom_settings` |Contains custom meta data about the job. | No
|
||||||
|
|
||||||
|
|`description` |An optional description of the job.
|
||||||
|
See <<ml-job-resource>>. | No
|
||||||
|
|
||||||
|
|`model_plot_config`: `enabled` |If true, enables calculation and storage of the
|
||||||
|
model bounds for each entity that is being analyzed.
|
||||||
|
See <<ml-apimodelplotconfig>>. | No
|
||||||
|
|
||||||
|
|`model_snapshot_retention_days` |The time in days that model snapshots are
|
||||||
|
retained for the job. See <<ml-job-resource>>. | Yes
|
||||||
|
|
||||||
|
|`renormalization_window_days` |Advanced configuration option. The period over
|
||||||
|
which adjustments to the score are applied, as new data is seen.
|
||||||
|
See <<ml-job-resource>>. | Yes
|
||||||
|
|
||||||
|
|`results_retention_days` |Advanced configuration option. The number of days
|
||||||
|
for which job results are retained. See <<ml-job-resource>>. | Yes
|
||||||
|
|
||||||
|
|=======================================================================
|
||||||
|
|
||||||
|
For those properties that have `Requires Restart` set to `Yes` in this table,
|
||||||
|
if the job is open when you make the update, you must stop the data feed, close
|
||||||
|
the job, then restart the data feed and open the job for the changes to take
|
||||||
|
effect.
|
||||||
|
|
||||||
|
//|`analysis_config`: `detectors`: `index` | A unique identifier of the
|
||||||
|
//detector. Matches the order of detectors returned by
|
||||||
|
//<<ml-get-job,GET job>>, starting from 0. | No
|
||||||
|
//|`analysis_config`: `detectors`: `detector_description` |A description of the
|
||||||
|
//detector. See <<ml-analysisconfig>>. | No
|
||||||
|
|
||||||
[NOTE]
|
[NOTE]
|
||||||
--
|
--
|
||||||
|
@ -43,9 +70,6 @@ of `hard_limit`, this means that it was unable to process some data. You might
|
||||||
want to re-run this job with an increased `model_memory_limit`.
|
want to re-run this job with an increased `model_memory_limit`.
|
||||||
--
|
--
|
||||||
|
|
||||||
`description`::
|
|
||||||
(string) An optional description of the job.
|
|
||||||
|
|
||||||
|
|
||||||
===== Authorization
|
===== Authorization
|
||||||
|
|
||||||
|
@ -55,32 +79,81 @@ For more information, see <<privileges-list-cluster>>.
|
||||||
|
|
||||||
===== Examples
|
===== Examples
|
||||||
|
|
||||||
The following example updates the `it-ops-kpi` job:
|
The following example updates the `it_ops_new_logs` job:
|
||||||
|
|
||||||
[source,js]
|
[source,js]
|
||||||
--------------------------------------------------
|
--------------------------------------------------
|
||||||
POST _xpack/ml/anomaly_detectors/it-ops-kpi/_update
|
POST _xpack/ml/anomaly_detectors/it_ops_new_logs/_update
|
||||||
{
|
{
|
||||||
"description":"New description",
|
"description":"An updated job",
|
||||||
|
"model_plot_config": {
|
||||||
|
"enabled": true
|
||||||
|
},
|
||||||
"analysis_limits": {
|
"analysis_limits": {
|
||||||
"model_memory_limit": 8192
|
"model_memory_limit": 1024
|
||||||
|
},
|
||||||
|
"renormalization_window_days": 30,
|
||||||
|
"background_persist_interval": "2h",
|
||||||
|
"model_snapshot_retention_days": 7,
|
||||||
|
"results_retention_days": 60,
|
||||||
|
"custom_settings": {
|
||||||
|
"custom_urls" : [{
|
||||||
|
"url_name" : "Lookup IP",
|
||||||
|
"url_value" : "http://geoiplookup.net/ip/$clientip$"
|
||||||
|
}]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
--------------------------------------------------
|
--------------------------------------------------
|
||||||
// CONSOLE
|
// CONSOLE
|
||||||
// TEST[skip:todo]
|
// TEST[skip:todo]
|
||||||
|
|
||||||
When the job is updated, you receive the following results:
|
When the job is updated, you receive a summary of the job configuration
|
||||||
|
information, including the updated property values. For example:
|
||||||
|
|
||||||
[source,js]
|
[source,js]
|
||||||
----
|
----
|
||||||
{
|
{
|
||||||
"job_id": "it-ops-kpi",
|
"job_id": "it_ops_new_logs",
|
||||||
"job_type": "anomaly_detector",
|
"job_type": "anomaly_detector",
|
||||||
"description": "New description",
|
"description": "An updated job",
|
||||||
...
|
"create_time": 1493678314204,
|
||||||
"analysis_limits": {
|
"finished_time": 1493678315850,
|
||||||
"model_memory_limit": 8192
|
"analysis_config": {
|
||||||
|
"bucket_span": "1800s",
|
||||||
|
"categorization_field_name": "message",
|
||||||
|
"detectors": [
|
||||||
|
{
|
||||||
|
"detector_description": "Unusual message counts",
|
||||||
|
"function": "count",
|
||||||
|
"by_field_name": "mlcategory",
|
||||||
|
"detector_rules": []
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"influencers": []
|
||||||
},
|
},
|
||||||
...
|
"analysis_limits": {
|
||||||
|
"model_memory_limit": 1024
|
||||||
|
},
|
||||||
|
"data_description": {
|
||||||
|
"time_field": "time",
|
||||||
|
"time_format": "epoch_ms"
|
||||||
|
},
|
||||||
|
"model_plot_config": {
|
||||||
|
"enabled": true
|
||||||
|
},
|
||||||
|
"renormalization_window_days": 30,
|
||||||
|
"background_persist_interval": "2h",
|
||||||
|
"model_snapshot_retention_days": 7,
|
||||||
|
"results_retention_days": 60,
|
||||||
|
"custom_settings": {
|
||||||
|
"custom_urls": [
|
||||||
|
{
|
||||||
|
"url_name": "Lookup IP",
|
||||||
|
"url_value": "http://geoiplookup.net/ip/$clientip$"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"model_snapshot_id": "1493678315",
|
||||||
|
"results_index_name": "shared"
|
||||||
}
|
}
|
||||||
----
|
----
|
||||||
|
|
Loading…
Reference in New Issue