mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-08 14:05:27 +00:00
984323783e
This change adds: - A new option, allow_lazy_open, to anomaly detection jobs - A new option, allow_lazy_start, to data frame analytics jobs Both work in the same way: they allow a job to be opened/started even if no ML node exists that can accommodate the job immediately. In this situation the job waits in the opening/starting state until ML node capacity is available. (The starting state for data frame analytics jobs is new in this change.) Additionally, the ML nightly maintenance tasks now creates audit warnings for ML jobs that are unassigned. This means that jobs that cannot be assigned to an ML node for a very long time will show a yellow warning triangle in the UI. A final change is that it is now possible to close a job that is not assigned to a node without using force. This is because previously jobs that were open but not assigned to a node were an aberration, whereas after this change they'll be relatively common.
182 lines
5.3 KiB
Plaintext
182 lines
5.3 KiB
Plaintext
[role="xpack"]
|
|
[testenv="platinum"]
|
|
[[ml-update-job]]
|
|
=== Update {anomaly-jobs} API
|
|
++++
|
|
<titleabbrev>Update jobs</titleabbrev>
|
|
++++
|
|
|
|
Updates certain properties of an {anomaly-job}.
|
|
|
|
[[ml-update-job-request]]
|
|
==== {api-request-title}
|
|
|
|
`POST _ml/anomaly_detectors/<job_id>/_update`
|
|
|
|
[[ml-update-job-prereqs]]
|
|
==== {api-prereq-title}
|
|
|
|
* If the {es} {security-features} are enabled, you must have `manage_ml` or
|
|
`manage` cluster privileges to use this API. See
|
|
<<security-privileges>>.
|
|
|
|
|
|
[[ml-update-job-path-parms]]
|
|
==== {api-path-parms-title}
|
|
|
|
`<job_id>`::
|
|
(Required, string) Identifier for the {anomaly-job}.
|
|
|
|
[[ml-update-job-request-body]]
|
|
==== {api-request-body-title}
|
|
|
|
The following properties can be updated after the job is created:
|
|
|
|
[cols="<,<,<",options="header",]
|
|
|=======================================================================
|
|
|Name |Description |Requires Restart
|
|
|
|
|`analysis_limits.model_memory_limit` |The approximate maximum amount of
|
|
memory resources required for analytical processing. See <<ml-apilimits>>. You
|
|
can update the `analysis_limits` only while the job is closed. The
|
|
`model_memory_limit` property value cannot be decreased below the current usage.
|
|
| Yes
|
|
|
|
|`background_persist_interval` |Advanced configuration option. The time between
|
|
each periodic persistence of the model. See <<ml-job-resource>>. | Yes
|
|
|
|
|`custom_settings` |Contains custom meta data about the job. | No
|
|
|
|
|`description` |A description of the job. See <<ml-job-resource>>. | No
|
|
|
|
|`detectors` |An array of detector update objects. | No
|
|
|
|
|`detector_index` |The identifier of the detector to update (integer).| No
|
|
|
|
|`detectors.description` |The new description for the detector.| No
|
|
|
|
|`detectors.custom_rules` |The new list of <<ml-detector-custom-rule, rules>>
|
|
for the detector. | No
|
|
|
|
|`groups` |A list of job groups. See <<ml-job-resource>>. | No
|
|
|
|
|`model_plot_config.enabled` |If true, enables calculation and storage of the
|
|
model bounds for each entity that is being analyzed.
|
|
See <<ml-apimodelplotconfig>>. | No
|
|
|
|
|`model_snapshot_retention_days` |The time in days that model snapshots are
|
|
retained for the job. See <<ml-job-resource>>. | Yes
|
|
|
|
|`renormalization_window_days` |Advanced configuration option. The period over
|
|
which adjustments to the score are applied, as new data is seen.
|
|
See <<ml-job-resource>>. | Yes
|
|
|
|
|`results_retention_days` |Advanced configuration option. The number of days
|
|
for which job results are retained. See <<ml-job-resource>>. | Yes
|
|
|
|
|`allow_lazy_open` |Advanced configuration option. Whether to allow the job to be
|
|
opened when no {ml} node has sufficient capacity. See <<ml-job-resource>>. | Yes
|
|
|
|
|=======================================================================
|
|
|
|
For those properties that have `Requires Restart` set to `Yes` in this table,
|
|
if the job is open when you make the update, you must stop the data feed, close
|
|
the job, then reopen the job and restart the data feed for the changes to take
|
|
effect.
|
|
|
|
[NOTE]
|
|
--
|
|
* If the `memory_status` property in the `model_size_stats` object has a value
|
|
of `hard_limit`, this means that it was unable to process some data. You might
|
|
want to re-run this job with an increased `model_memory_limit`.
|
|
--
|
|
|
|
|
|
[[ml-update-job-example]]
|
|
==== {api-examples-title}
|
|
|
|
The following example updates the `total-requests` job:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
POST _ml/anomaly_detectors/total-requests/_update
|
|
{
|
|
"description":"An updated job",
|
|
"groups": ["group1","group2"],
|
|
"model_plot_config": {
|
|
"enabled": true
|
|
},
|
|
"analysis_limits": {
|
|
"model_memory_limit": "1024mb"
|
|
},
|
|
"renormalization_window_days": 30,
|
|
"background_persist_interval": "2h",
|
|
"model_snapshot_retention_days": 7,
|
|
"results_retention_days": 60,
|
|
"custom_settings": {
|
|
"custom_urls" : [{
|
|
"url_name" : "Lookup IP",
|
|
"url_value" : "http://geoiplookup.net/ip/$clientip$"
|
|
}]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[skip:setup:server_metrics_job]
|
|
|
|
When the {anomaly-job} is updated, you receive a summary of the job
|
|
configuration information, including the updated property values. For example:
|
|
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"job_id": "total-requests",
|
|
"job_type": "anomaly_detector",
|
|
"job_version": "7.0.0-alpha1",
|
|
"groups": [
|
|
"group1",
|
|
"group2"
|
|
],
|
|
"description": "An updated job",
|
|
"create_time": 1518808660505,
|
|
"analysis_config": {
|
|
"bucket_span": "10m",
|
|
"detectors": [
|
|
{
|
|
"detector_description": "Sum of total",
|
|
"function": "sum",
|
|
"field_name": "total",
|
|
"detector_index": 0
|
|
}
|
|
],
|
|
"influencers": []
|
|
},
|
|
"analysis_limits": {
|
|
"model_memory_limit": "1024mb",
|
|
"categorization_examples_limit": 4
|
|
},
|
|
"data_description": {
|
|
"time_field": "timestamp",
|
|
"time_format": "epoch_ms"
|
|
},
|
|
"model_plot_config": {
|
|
"enabled": true
|
|
},
|
|
"renormalization_window_days": 30,
|
|
"background_persist_interval": "2h",
|
|
"model_snapshot_retention_days": 7,
|
|
"results_retention_days": 60,
|
|
"custom_settings": {
|
|
"custom_urls": [
|
|
{
|
|
"url_name": "Lookup IP",
|
|
"url_value": "http://geoiplookup.net/ip/$clientip$"
|
|
}
|
|
]
|
|
},
|
|
"results_index_name": "shared",
|
|
"allow_lazy_open": false
|
|
}
|
|
----
|
|
// TESTRESPONSE[s/"job_version": "7.0.0-alpha1"/"job_version": $body.job_version/]
|
|
// TESTRESPONSE[s/"create_time": 1518808660505/"create_time": $body.create_time/]
|