2017-06-19 21:23:58 -04:00
|
|
|
[role="xpack"]
|
2018-08-31 19:49:24 -04:00
|
|
|
[testenv="platinum"]
|
2017-04-04 18:26:39 -04:00
|
|
|
[[ml-update-job]]
|
2019-08-06 14:05:01 -04:00
|
|
|
=== Update {anomaly-jobs} API
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2018-12-20 13:23:28 -05:00
|
|
|
<titleabbrev>Update jobs</titleabbrev>
|
2017-12-14 13:52:49 -05:00
|
|
|
++++
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-08-06 14:05:01 -04:00
|
|
|
Updates certain properties of an {anomaly-job}.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-update-job-request]]
|
|
|
|
==== {api-request-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-12-07 15:34:11 -05:00
|
|
|
`POST _ml/anomaly_detectors/<job_id>/_update`
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 16:58:42 -04:00
|
|
|
[[ml-update-job-prereqs]]
|
|
|
|
==== {api-prereq-title}
|
|
|
|
|
|
|
|
* If the {es} {security-features} are enabled, you must have `manage_ml` or
|
|
|
|
`manage` cluster privileges to use this API. See
|
|
|
|
{stack-ov}/security-privileges.html[Security privileges].
|
|
|
|
|
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-update-job-path-parms]]
|
|
|
|
==== {api-path-parms-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-07-12 11:26:31 -04:00
|
|
|
`<job_id>`::
|
2019-08-06 14:05:01 -04:00
|
|
|
(Required, string) Identifier for the {anomaly-job}.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-update-job-request-body]]
|
|
|
|
==== {api-request-body-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
|
|
|
The following properties can be updated after the job is created:
|
|
|
|
|
2017-05-02 18:34:30 -04:00
|
|
|
[cols="<,<,<",options="header",]
|
|
|
|
|=======================================================================
|
|
|
|
|Name |Description |Requires Restart
|
|
|
|
|
|
|
|
|`analysis_limits`: `model_memory_limit` |The approximate maximum amount of
|
2017-07-10 11:50:38 -04:00
|
|
|
memory resources required for analytical processing. See <<ml-apilimits>>. | Yes
|
2017-05-02 18:34:30 -04:00
|
|
|
|
|
|
|
|`background_persist_interval` |Advanced configuration option. The time between
|
|
|
|
each periodic persistence of the model. See <<ml-job-resource>>. | Yes
|
|
|
|
|
|
|
|
|`custom_settings` |Contains custom meta data about the job. | No
|
|
|
|
|
2017-08-17 15:52:29 -04:00
|
|
|
|`description` |A description of the job. See <<ml-job-resource>>. | No
|
|
|
|
|
2018-07-25 11:10:32 -04:00
|
|
|
|`detectors` |An array of <<ml-detector-update, detector update objects>>. | No
|
|
|
|
|
2017-08-17 15:52:29 -04:00
|
|
|
|`groups` |A list of job groups. See <<ml-job-resource>>. | No
|
2017-05-02 18:34:30 -04:00
|
|
|
|
|
|
|
|`model_plot_config`: `enabled` |If true, enables calculation and storage of the
|
|
|
|
model bounds for each entity that is being analyzed.
|
|
|
|
See <<ml-apimodelplotconfig>>. | No
|
|
|
|
|
|
|
|
|`model_snapshot_retention_days` |The time in days that model snapshots are
|
|
|
|
retained for the job. See <<ml-job-resource>>. | Yes
|
|
|
|
|
|
|
|
|`renormalization_window_days` |Advanced configuration option. The period over
|
|
|
|
which adjustments to the score are applied, as new data is seen.
|
|
|
|
See <<ml-job-resource>>. | Yes
|
|
|
|
|
|
|
|
|`results_retention_days` |Advanced configuration option. The number of days
|
|
|
|
for which job results are retained. See <<ml-job-resource>>. | Yes
|
|
|
|
|
|
|
|
|=======================================================================
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2017-05-02 18:34:30 -04:00
|
|
|
For those properties that have `Requires Restart` set to `Yes` in this table,
|
|
|
|
if the job is open when you make the update, you must stop the data feed, close
|
|
|
|
the job, then restart the data feed and open the job for the changes to take
|
|
|
|
effect.
|
|
|
|
|
2017-04-24 13:46:17 -04:00
|
|
|
[NOTE]
|
|
|
|
--
|
|
|
|
* You can update the `analysis_limits` only while the job is closed.
|
2018-03-08 09:14:18 -05:00
|
|
|
* The `model_memory_limit` property value cannot be decreased below the current usage.
|
2017-04-24 13:46:17 -04:00
|
|
|
* If the `memory_status` property in the `model_size_stats` object has a value
|
|
|
|
of `hard_limit`, this means that it was unable to process some data. You might
|
|
|
|
want to re-run this job with an increased `model_memory_limit`.
|
|
|
|
--
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-07-25 11:10:32 -04:00
|
|
|
[[ml-detector-update]]
|
2019-08-06 14:05:01 -04:00
|
|
|
==== Detector update objects
|
2018-07-25 11:10:32 -04:00
|
|
|
|
|
|
|
A detector update object has the following properties:
|
|
|
|
|
|
|
|
`detector_index`::
|
|
|
|
(integer) The identifier of the detector to update.
|
|
|
|
|
|
|
|
`description`::
|
|
|
|
(string) The new description for the detector.
|
|
|
|
|
|
|
|
`custom_rules`::
|
|
|
|
(array) The new list of <<ml-detector-custom-rule, rules>> for the detector.
|
|
|
|
|
|
|
|
No other detector property can be updated.
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-06-27 12:42:47 -04:00
|
|
|
[[ml-update-job-example]]
|
|
|
|
==== {api-examples-title}
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2018-02-20 14:08:37 -05:00
|
|
|
The following example updates the `total-requests` job:
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-09-09 12:35:50 -04:00
|
|
|
[source,console]
|
2017-04-04 18:26:39 -04:00
|
|
|
--------------------------------------------------
|
2018-12-07 15:34:11 -05:00
|
|
|
POST _ml/anomaly_detectors/total-requests/_update
|
2017-04-04 18:26:39 -04:00
|
|
|
{
|
2017-05-02 18:34:30 -04:00
|
|
|
"description":"An updated job",
|
2017-08-17 15:52:29 -04:00
|
|
|
"groups": ["group1","group2"],
|
2017-05-02 18:34:30 -04:00
|
|
|
"model_plot_config": {
|
|
|
|
"enabled": true
|
|
|
|
},
|
|
|
|
"analysis_limits": {
|
2017-07-06 10:40:19 -04:00
|
|
|
"model_memory_limit": "1024mb"
|
2017-05-02 18:34:30 -04:00
|
|
|
},
|
|
|
|
"renormalization_window_days": 30,
|
|
|
|
"background_persist_interval": "2h",
|
|
|
|
"model_snapshot_retention_days": 7,
|
|
|
|
"results_retention_days": 60,
|
|
|
|
"custom_settings": {
|
|
|
|
"custom_urls" : [{
|
|
|
|
"url_name" : "Lookup IP",
|
|
|
|
"url_value" : "http://geoiplookup.net/ip/$clientip$"
|
|
|
|
}]
|
|
|
|
}
|
2017-04-04 18:26:39 -04:00
|
|
|
}
|
|
|
|
--------------------------------------------------
|
2018-08-31 19:49:24 -04:00
|
|
|
// TEST[skip:setup:server_metrics_job]
|
2017-04-04 18:26:39 -04:00
|
|
|
|
2019-08-06 14:05:01 -04:00
|
|
|
When the {anomaly-job} is updated, you receive a summary of the job
|
|
|
|
configuration information, including the updated property values. For example:
|
2017-05-02 18:34:30 -04:00
|
|
|
|
2019-09-06 16:09:09 -04:00
|
|
|
[source,console-result]
|
2017-04-04 18:26:39 -04:00
|
|
|
----
|
|
|
|
{
|
2018-02-20 14:08:37 -05:00
|
|
|
"job_id": "total-requests",
|
2017-04-11 21:52:47 -04:00
|
|
|
"job_type": "anomaly_detector",
|
2017-08-17 15:52:29 -04:00
|
|
|
"job_version": "7.0.0-alpha1",
|
|
|
|
"groups": [
|
|
|
|
"group1",
|
|
|
|
"group2"
|
|
|
|
],
|
2017-05-02 18:34:30 -04:00
|
|
|
"description": "An updated job",
|
2018-02-20 14:08:37 -05:00
|
|
|
"create_time": 1518808660505,
|
2017-05-02 18:34:30 -04:00
|
|
|
"analysis_config": {
|
2018-02-20 14:08:37 -05:00
|
|
|
"bucket_span": "10m",
|
2017-05-02 18:34:30 -04:00
|
|
|
"detectors": [
|
|
|
|
{
|
2018-02-20 14:08:37 -05:00
|
|
|
"detector_description": "Sum of total",
|
|
|
|
"function": "sum",
|
|
|
|
"field_name": "total",
|
2017-06-02 05:26:01 -04:00
|
|
|
"detector_index": 0
|
2017-05-02 18:34:30 -04:00
|
|
|
}
|
|
|
|
],
|
|
|
|
"influencers": []
|
|
|
|
},
|
2017-04-04 18:26:39 -04:00
|
|
|
"analysis_limits": {
|
[ML] Set explicit defaults to AnalysisLimits (elastic/x-pack-elasticsearch#4015)
Analysis limits contain settings that affect the resources
used by ML jobs. Those limits always take place. However,
explictly setting them is not required as they have reasonable
defaults. For a long time those defaults lived on the c++ side.
The job could just not have any explicit limits and that meant
defaults would be used at the c++ side. This has the disadvantage
that it is not obvious to the users what these settings are set to.
Additionally, users might not be aware of the settings existence.
On top of that, since 6.1, the default model_memory_limit was lowered
from 4GB to 1GB. For BWC, this meant that jobs where model_memory_limit
is null, the default of 4GB applies. Jobs that were created from 6.1
onwards, contain an explicit setting for model_memory_limit, which is
1GB unless the user sets it differently. This adds additional confusion.
This commit makes analysis limits an always explicit setting on the job.
Regardless of whether the user sets custom limits or not, the job object
(and response) will contain the full analysis limits values.
The possibilities for interpretation of missing values are:
- the entire analysis_limits is null: this may only happen for jobs
created prior to 6.1. Thus we set the model_memory_limit to 4GB.
- analysis_limits are non-null but model_memory_limit is: this also
may only happen for jobs prior to 6.1. Again, we set memory limit to
4GB.
- model_memory_limit is non-null: this either means the user set an
explicit value or the job was created from 6.1 onwards and it has
the explicit default of 1GB. We simply keep the given value.
For categorization_examples_limit the default has always been 4, so
we fill that in when it's missing.
Finally, note that we still need to handle potential null values
for the situation of a mixed cluster.
Original commit: elastic/x-pack-elasticsearch@5b6994ef750298a829dd2995664470cd4cc95e07
2018-02-27 12:49:05 -05:00
|
|
|
"model_memory_limit": "1024mb",
|
|
|
|
"categorization_examples_limit": 4
|
2017-05-02 18:34:30 -04:00
|
|
|
},
|
|
|
|
"data_description": {
|
2018-02-20 14:08:37 -05:00
|
|
|
"time_field": "timestamp",
|
2017-05-02 18:34:30 -04:00
|
|
|
"time_format": "epoch_ms"
|
|
|
|
},
|
|
|
|
"model_plot_config": {
|
|
|
|
"enabled": true
|
|
|
|
},
|
|
|
|
"renormalization_window_days": 30,
|
|
|
|
"background_persist_interval": "2h",
|
|
|
|
"model_snapshot_retention_days": 7,
|
|
|
|
"results_retention_days": 60,
|
|
|
|
"custom_settings": {
|
|
|
|
"custom_urls": [
|
|
|
|
{
|
|
|
|
"url_name": "Lookup IP",
|
|
|
|
"url_value": "http://geoiplookup.net/ip/$clientip$"
|
|
|
|
}
|
|
|
|
]
|
2017-04-11 21:52:47 -04:00
|
|
|
},
|
2017-05-02 18:34:30 -04:00
|
|
|
"results_index_name": "shared"
|
2017-04-04 18:26:39 -04:00
|
|
|
}
|
|
|
|
----
|
2018-02-20 14:08:37 -05:00
|
|
|
// TESTRESPONSE[s/"job_version": "7.0.0-alpha1"/"job_version": $body.job_version/]
|
2018-12-07 15:34:11 -05:00
|
|
|
// TESTRESPONSE[s/"create_time": 1518808660505/"create_time": $body.create_time/]
|