From af470026546886f15f25e80afc45786bcb002c97 Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Mon, 24 Apr 2017 10:46:17 -0700 Subject: [PATCH] [DOCS] Edits for ML doc changes in elastic/x-pack-elasticsearch#1169 (elastic/x-pack-elasticsearch#1177) Original commit: elastic/x-pack-elasticsearch@84f8b4b9bd00c9fea6c7cd9610fc46076bf07e3e --- docs/en/rest-api/ml/get-bucket.asciidoc | 7 +- .../rest-api/ml/get-datafeed-stats.asciidoc | 3 +- docs/en/rest-api/ml/get-datafeed.asciidoc | 3 +- docs/en/rest-api/ml/get-influencer.asciidoc | 8 ++- docs/en/rest-api/ml/get-job-stats.asciidoc | 5 +- docs/en/rest-api/ml/get-job.asciidoc | 5 +- docs/en/rest-api/ml/get-record.asciidoc | 7 +- docs/en/rest-api/ml/jobcounts.asciidoc | 15 +--- docs/en/rest-api/ml/jobresource.asciidoc | 70 +++++++------------ docs/en/rest-api/ml/preview-datafeed.asciidoc | 5 +- docs/en/rest-api/ml/resultsresource.asciidoc | 4 +- docs/en/rest-api/ml/snapshotresource.asciidoc | 3 +- docs/en/rest-api/ml/update-job.asciidoc | 20 ++++-- docs/en/rest-api/ml/update-snapshot.asciidoc | 6 +- 14 files changed, 76 insertions(+), 85 deletions(-) diff --git a/docs/en/rest-api/ml/get-bucket.asciidoc b/docs/en/rest-api/ml/get-bucket.asciidoc index 4a98b920758..9635664ea4c 100644 --- a/docs/en/rest-api/ml/get-bucket.asciidoc +++ b/docs/en/rest-api/ml/get-bucket.asciidoc @@ -39,15 +39,16 @@ roles provide these privileges. For more information, see `end`:: (string) Returns buckets with timestamps earlier than this time. +`exclude_interim`:: + (boolean) If true, the output excludes interim results. + By default, interim results are included. + `expand`:: (boolean) If true, the output includes anomaly records. `from`:: (integer) Skips the specified number of buckets. -`exclude_interim`:: - (boolean) If true, the output excludes interim results. These are included by default. - `size`:: (integer) Specifies the maximum number of buckets to obtain. diff --git a/docs/en/rest-api/ml/get-datafeed-stats.asciidoc b/docs/en/rest-api/ml/get-datafeed-stats.asciidoc index a82d4f5ce84..574fbd9bc2b 100644 --- a/docs/en/rest-api/ml/get-datafeed-stats.asciidoc +++ b/docs/en/rest-api/ml/get-datafeed-stats.asciidoc @@ -23,7 +23,8 @@ privileges to use this API. For more information, see < `feed_id`:: (string) Identifier for the data feed. - Does not support wildcards, however you may specify `_all` to get information about all data feeds. + This parameter does not support wildcards, but you can specify `_all` or + omit the `feed_id` to get information about all data feeds. ===== Results diff --git a/docs/en/rest-api/ml/get-datafeed.asciidoc b/docs/en/rest-api/ml/get-datafeed.asciidoc index 661170a1dba..a2d3b4c3b81 100644 --- a/docs/en/rest-api/ml/get-datafeed.asciidoc +++ b/docs/en/rest-api/ml/get-datafeed.asciidoc @@ -22,7 +22,8 @@ privileges to use this API. For more information, see < `feed_id`:: (string) Identifier for the data feed. - Does not support wildcards, however you may specify `_all` or leave blank to get information about all data feeds. + This parameter does not support wildcards, but you can specify `_all` or + omit the `feed_id` to get information about all data feeds. ===== Results diff --git a/docs/en/rest-api/ml/get-influencer.asciidoc b/docs/en/rest-api/ml/get-influencer.asciidoc index d0192bc8caa..3a9cc7206e6 100644 --- a/docs/en/rest-api/ml/get-influencer.asciidoc +++ b/docs/en/rest-api/ml/get-influencer.asciidoc @@ -31,12 +31,14 @@ roles provide these privileges. For more information, see `end`:: (string) Returns influencers with timestamps earlier than this time. +`exclude_interim`:: + (boolean) If true, the output excludes interim results. + By default, interim results are included. + + `from`:: (integer) Skips the specified number of influencers. -`exclude_interim`:: - (boolean) If true, the output excludes interim results. These are included by default. - `influencer_score`:: (double) Returns influencers with anomaly scores higher than this value. diff --git a/docs/en/rest-api/ml/get-job-stats.asciidoc b/docs/en/rest-api/ml/get-job-stats.asciidoc index d32409e46f9..3cf1289ee65 100644 --- a/docs/en/rest-api/ml/get-job-stats.asciidoc +++ b/docs/en/rest-api/ml/get-job-stats.asciidoc @@ -19,8 +19,9 @@ privileges to use this API. For more information, see < ===== Path Parameters `job_id`:: - (string) A required identifier for the job. - Does not support wildcards, however you may specify `_all` to get information about all jobs. + (string) A required identifier for the job. + This parameter does not support wildcards, but you can specify `_all` or omit + the `job_id` to get information about all jobs. ===== Results diff --git a/docs/en/rest-api/ml/get-job.asciidoc b/docs/en/rest-api/ml/get-job.asciidoc index ee5e7142278..2d0f9c42dd0 100644 --- a/docs/en/rest-api/ml/get-job.asciidoc +++ b/docs/en/rest-api/ml/get-job.asciidoc @@ -19,8 +19,9 @@ privileges to use this API. For more information, see < ===== Path Parameters `job_id`:: - (string) Identifier for the job. - Does not support wildcards, however you may specify `_all` or leave blank to get information about all jobs. + (string) Identifier for the job. + This parameter does not support wildcards, but you can specify `_all` or omit + the `job_id` to get information about all jobs. ===== Results diff --git a/docs/en/rest-api/ml/get-record.asciidoc b/docs/en/rest-api/ml/get-record.asciidoc index 84ed7a39349..101c7064642 100644 --- a/docs/en/rest-api/ml/get-record.asciidoc +++ b/docs/en/rest-api/ml/get-record.asciidoc @@ -30,12 +30,13 @@ roles provide these privileges. For more information, see `end`:: (string) Returns records with timestamps earlier than this time. +`exclude_interim`:: + (boolean) If true, the output excludes interim results. + By default, interim results are included. + `from`:: (integer) Skips the specified number of records. -`exclude_interim`:: - (boolean) If true, the output excludes interim results. These are included by default. - `record_score`:: (double) Returns records with anomaly scores higher than this value. diff --git a/docs/en/rest-api/ml/jobcounts.asciidoc b/docs/en/rest-api/ml/jobcounts.asciidoc index ceb77939516..daa8f03dd9d 100644 --- a/docs/en/rest-api/ml/jobcounts.asciidoc +++ b/docs/en/rest-api/ml/jobcounts.asciidoc @@ -96,14 +96,11 @@ or old results are deleted, the job counts are not reset. configured to analyze. Records with missing fields are still processed because it is possible that not all fields are missing. The value of `processed_record_count` includes this count. + -+ --- + NOTE: If you are using data feeds or posting data to the job in JSON format, a high `missing_field_count` is often not an indication of data issues. It is not necessarily a cause for concern. --- - `out_of_order_timestamp_count`:: (long) The number of records that are out of time sequence and outside of the latency window. This information is applicable only when @@ -171,19 +168,13 @@ The `model_size_stats` object has the following properties: `total_by_field_count`:: (long) The number of `by` field values that were analyzed by the models. -+ --- -NOTE: The `by` field values are counted separately for each detector and partition. --- +NOTE: The `by` field values are counted separately for each detector and partition. `total_over_field_count`:: (long) The number of `over` field values that were analyzed by the models. -+ --- -NOTE: The `over` field values are counted separately for each detector and partition. --- +NOTE: The `over` field values are counted separately for each detector and partition. `total_partition_field_count`:: (long) The number of `partition` field values that were analyzed by the models. diff --git a/docs/en/rest-api/ml/jobresource.asciidoc b/docs/en/rest-api/ml/jobresource.asciidoc index 17127ae3cd4..7c7b7a1bb2b 100644 --- a/docs/en/rest-api/ml/jobresource.asciidoc +++ b/docs/en/rest-api/ml/jobresource.asciidoc @@ -13,11 +13,14 @@ A job resource has the following properties: See <>. `background_persist_interval`:: - (time units) Advanced configuration option. - The time between each periodic persistence of the model. - The default value is a randomized value between 3 to 4 hours which avoid all jobs persisting at exactly the same time. - For very large models (several GB), persistence could take 10-20 minutes, so please do not set this value too low. - The smallest allowed value is 1 hour. + (time units) Advanced configuration option. + The time between each periodic persistence of the model. + The default value is a randomized value between 3 to 4 hours, which avoids + all jobs persisting at exactly the same time. The smallest allowed value is + 1 hour. + + +TIP: For very large models (several GB), persistence could take 10-20 minutes, +so do not set the `background_persist_interval` value too low. `create_time`:: (string) The time the job was created, in ISO 8601 format. @@ -36,7 +39,7 @@ A job resource has the following properties: `job_id`:: (string) The unique identifier for the job. - + `job_type`:: (string) Reserved for future use, currently set to `anomaly_detector`. @@ -63,10 +66,11 @@ A job resource has the following properties: which corresponds to the index name `.ml-anomalies-shared` `results_retention_days`:: - (long) Advanced configuration option. - The number of days for which job results are retained. - Once per day at 00:30 (server time), results older than this period will be deleted from Elasticsearch. - The default value is null, i.e. results are retained. + (long) Advanced configuration option. + The number of days for which job results are retained. + Once per day at 00:30 (server time), results older than this period are + deleted from Elasticsearch. The default value is null, which means results + are retained. [[ml-analysisconfig]] ===== Analysis Configuration Objects @@ -95,12 +99,10 @@ An analysis configuration object has the following properties: (array) An array of detector configuration objects, which describe the anomaly detectors that are used in the job. See <>. + -+ --- -NOTE: If the `detectors` array does not contain at least one detector, no analysis can occur -and an error is returned. --- +NOTE: If the `detectors` array does not contain at least one detector, +no analysis can occur and an error is returned. + `influencers`:: (array of strings) A comma separated list of influencer field names. Typically these can be the by, over, or partition fields that are used in the @@ -112,12 +114,10 @@ and an error is returned. `latency`:: (unsigned integer) The size of the window, in seconds, in which to expect data that is out of time order. The default value is 0 (no latency). + -+ --- + NOTE: Latency is only applicable when you send data by using the <> API. --- `multivariate_by_fields`:: (boolean) If set to `true`, the analysis will automatically find correlations between metrics for a given `by` field value and report anomalies when those @@ -126,24 +126,20 @@ the <> API. correlation occurs because they are running a load-balanced application. If you enable this property, then anomalies will be reported when, for example, CPU usage on host A is high and the value of CPU usage on host B is low. - That is to say, you'll see an anomaly when the CPU of host A is unusual given the CPU of host B. + -+ --- + That is to say, you'll see an anomaly when the CPU of host A is unusual given + the CPU of host B. + + NOTE: To use the `multivariate_by_fields` property, you must also specify `by_field_name` in your detector. --- - `summary_count_field_name`:: (string) If not null, the data fed to the job is expected to be pre-summarized. This property value is the name of the field that contains the count of raw data points that have been summarized. The same `summary_count_field_name` applies to all detectors in the job. + -+ --- -NOTE: The `summary_count_field_name` property cannot be used with the `metric` function. --- +NOTE: The `summary_count_field_name` property cannot be used with the `metric` +function. //// LEAVE UNDOCUMENTED @@ -181,11 +177,8 @@ Each detector has the following properties: `field_name`:: (string) The field that the detector uses in the function. If you use an event rate function such as `count` or `rare`, do not specify this field. + -+ --- -NOTE: The `field_name` cannot contain double quotes or backslashes. --- +NOTE: The `field_name` cannot contain double quotes or backslashes. `function` (required):: (string) The analysis function that is used. @@ -203,13 +196,10 @@ NOTE: The `field_name` cannot contain double quotes or backslashes. `use_null`:: (boolean) Defines whether a new series is used as the null series when there is no value for the by or partition fields. The default value is `false`. + -+ --- + IMPORTANT: Field names are case sensitive, for example a field named 'Bytes' is different from one named 'bytes'. --- - //// LEAVE UNDOCUMENTED `detector_rules`:: @@ -242,16 +232,13 @@ A data description object has the following properties: since 1 Jan 1970). The value `epoch_ms` indicates that time is measured in milliseconds since the epoch. The `epoch` and `epoch_ms` time formats accept either integer or real values. + -+ --- + NOTE: Custom patterns must conform to the Java `DateTimeFormatter` class. When you use date-time formatting patterns, it is recommended that you provide the full date, time and time zone. For example: `yyyy-MM-dd'T'HH:mm:ssX`. If the pattern that you specify is not sufficient to produce a complete timestamp, job creation fails. --- - [float] [[ml-apilimits]] @@ -269,11 +256,8 @@ The `analysis_limits` object has the following properties: in the results data store. The default value is 4. If you increase this value, more examples are available, however it requires that you have more storage available. If you set this value to `0`, no examples are stored. + -+ --- -NOTE: The `categorization_examples_limit` only applies to analysis that uses categorization. --- +NOTE: The `categorization_examples_limit` only applies to analysis that uses categorization. `model_memory_limit`:: (long) The maximum amount of memory, in MiB, that the mathematical models can use. diff --git a/docs/en/rest-api/ml/preview-datafeed.asciidoc b/docs/en/rest-api/ml/preview-datafeed.asciidoc index ef461a88f8a..6bd30083e73 100644 --- a/docs/en/rest-api/ml/preview-datafeed.asciidoc +++ b/docs/en/rest-api/ml/preview-datafeed.asciidoc @@ -11,8 +11,9 @@ The preview data feed API enables you to preview a data feed. ===== Description -The API returns the first "page" of results from the `search` created using the current data feed settings. -This shows the structure of the data that will be passed to the anomaly detection engine. +The API returns the first "page" of results from the `search` that is created +by using the current data feed settings. This preview shows the structure of +the data that will be passed to the anomaly detection engine. You must have `monitor_ml`, `monitor`, `manage_ml`, or `manage` cluster privileges to use this API. For more information, see <>. diff --git a/docs/en/rest-api/ml/resultsresource.asciidoc b/docs/en/rest-api/ml/resultsresource.asciidoc index f3f095111af..1ba5b6c8e0d 100644 --- a/docs/en/rest-api/ml/resultsresource.asciidoc +++ b/docs/en/rest-api/ml/resultsresource.asciidoc @@ -313,12 +313,10 @@ A bucket resource has the following properties: `timestamp`:: (date) The start time of the bucket, specified in ISO 8601 format. For example, 1454020800000. This timestamp uniquely identifies the bucket. + -+ --- + NOTE: Events that occur exactly at the timestamp of the bucket are included in the results for the bucket. --- [float] [[ml-results-categories]] diff --git a/docs/en/rest-api/ml/snapshotresource.asciidoc b/docs/en/rest-api/ml/snapshotresource.asciidoc index 61dd65711f9..2f989c9e947 100644 --- a/docs/en/rest-api/ml/snapshotresource.asciidoc +++ b/docs/en/rest-api/ml/snapshotresource.asciidoc @@ -3,7 +3,8 @@ ==== Model Snapshot Resources Model snapshots are saved to disk periodically. -By default, this is occurs approximately every 3 hours to 4 hours and is configurable using the setting `background_persist_interval`. +By default, this is occurs approximately every 3 hours to 4 hours and is +configurable with the `background_persist_interval` property. By default, model snapshots are retained for one day. You can change this behavior by updating the `model_snapshot_retention_days` for the job. diff --git a/docs/en/rest-api/ml/update-job.asciidoc b/docs/en/rest-api/ml/update-job.asciidoc index a14e53258f1..b7890702337 100644 --- a/docs/en/rest-api/ml/update-job.asciidoc +++ b/docs/en/rest-api/ml/update-job.asciidoc @@ -26,16 +26,22 @@ The following properties can be updated after the job is created: `analysis_config`:: (object) The analysis configuration, which specifies how to analyze the data. - See <>. In particular, the following properties can be updated: `categorization_filters`, `detector_description`, TBD. + See <>. In particular, + the following properties can be updated: `categorization_filters`, + `detector_description`, TBD. `analysis_limits`:: (object) Specifies runtime limits for the job. - See <>. NOTE: - * You can update the `analysis_limits` only while the job is closed. - * The `model_memory_limit` property value cannot be decreased. - * If the `memory_status` property in the `model_size_stats` object has a value of `hard_limit`, - this means that it was unable to process some data. You may wish to re-run this job - with an increased `model_memory_limit`. + See <>. + +[NOTE] +-- +* You can update the `analysis_limits` only while the job is closed. +* The `model_memory_limit` property value cannot be decreased. +* If the `memory_status` property in the `model_size_stats` object has a value +of `hard_limit`, this means that it was unable to process some data. You might +want to re-run this job with an increased `model_memory_limit`. +-- `description`:: (string) An optional description of the job. diff --git a/docs/en/rest-api/ml/update-snapshot.asciidoc b/docs/en/rest-api/ml/update-snapshot.asciidoc index 6aaf3014290..56a1b45f2fc 100644 --- a/docs/en/rest-api/ml/update-snapshot.asciidoc +++ b/docs/en/rest-api/ml/update-snapshot.asciidoc @@ -32,10 +32,12 @@ For more information, see <>. The following properties can be updated after the model snapshot is created: `description`:: - (string) An optional description of the model snapshot. E.g. "Before black friday" + (string) An optional description of the model snapshot. For example, + "Before black friday". `retain`:: - (boolean) If true, this snapshot will not be deleted during automatic cleanup of snapshots older than `model_snapshot_retention_days`. + (boolean) If true, this snapshot will not be deleted during automatic cleanup + of snapshots older than `model_snapshot_retention_days`. Note that this snapshot will still be deleted when the job is deleted. The default value is false.