[role="xpack"] [testenv="platinum"] [[ml-get-job-stats]] === Get {anomaly-job} statistics API ++++ Get job statistics ++++ Retrieves usage information for {anomaly-jobs}. [[ml-get-job-stats-request]] ==== {api-request-title} `GET _ml/anomaly_detectors//_stats` `GET _ml/anomaly_detectors/,/_stats` + `GET _ml/anomaly_detectors/_stats` + `GET _ml/anomaly_detectors/_all/_stats` [[ml-get-job-stats-prereqs]] ==== {api-prereq-title} * If the {es} {security-features} are enabled, you must have `monitor_ml`, `monitor`, `manage_ml`, or `manage` cluster privileges to use this API. See <>. [[ml-get-job-stats-desc]] ==== {api-description-title} You can get statistics for multiple {anomaly-jobs} in a single API request by using a group name, a comma-separated list of jobs, or a wildcard expression. You can get statistics for all {anomaly-jobs} by using `_all`, by specifying `*` as the ``, or by omitting the ``. IMPORTANT: This API returns a maximum of 10,000 jobs. [[ml-get-job-stats-path-parms]] ==== {api-path-parms-title} ``:: (Optional, string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection-default] [[ml-get-job-stats-query-parms]] ==== {api-query-parms-title} `allow_no_jobs`:: (Optional, boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=allow-no-jobs] [[ml-get-job-stats-results]] ==== {api-response-body-title} The API returns the following information about the operational progress of a job: `assignment_explanation`:: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=assignment-explanation-anomaly-jobs] [[datacounts]]`data_counts`:: (object) An object that describes the quantity of input to the job and any related error counts. The `data_count` values are cumulative for the lifetime of a job. If a model snapshot is reverted or old results are deleted, the job counts are not reset. `data_counts`.`bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-count-anomaly-jobs] `data_counts`.`earliest_record_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=earliest-record-timestamp] `data_counts`.`empty_bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=empty-bucket-count] `data_counts`.`input_bytes`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=input-bytes] `data_counts`.`input_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=input-field-count] `data_counts`.`input_record_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=input-record-count] `data_counts`.`invalid_date_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=invalid-date-count] `data_counts`.`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] `data_counts`.`last_data_time`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=last-data-time] `data_counts`.`latest_empty_bucket_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=latest-empty-bucket-timestamp] `data_counts`.`latest_record_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=latest-record-timestamp] `data_counts`.`latest_sparse_bucket_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=latest-sparse-record-timestamp] `data_counts`.`missing_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=missing-field-count] + The value of `processed_record_count` includes this count. `data_counts`.`out_of_order_timestamp_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=out-of-order-timestamp-count] `data_counts`.`processed_field_count`::: include::{docdir}/ml/ml-shared.asciidoc[tag=processed-field-count] `data_counts`.`processed_record_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=processed-record-count] `data_counts`.`sparse_bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=sparse-bucket-count] [[forecastsstats]]`forecasts_stats`:: (object) An object that provides statistical information about forecasts belonging to this job. Some statistics are omitted if no forecasts have been made. It has the following properties: + -- NOTE: `memory_bytes`, `records`, `processing_time_ms` and `status` require at least one forecast. Otherwise, these fields are omitted. -- `forecasts_stats`.`forecasted_jobs`::: (long) A value of `0` indicates that forecasts do not exist for this job. A value of `1` indicates that at least one forecast exists. `forecasts_stats`.`memory_bytes`::: (object) The `avg`, `min`, `max` and `total` memory usage in bytes for forecasts related to this job. If there are no forecasts, this property is omitted. `forecasts_stats`.`processing_time_ms`::: (object) Statistics about the forecast runtime in milliseconds: minimum, maximum, average and total. `forecasts_stats`.`records`::: (object) The `avg`, `min`, `max` and `total` number of `model_forecast` documents written for forecasts related to this job. If there are no forecasts, this property is omitted. `forecasts_stats`.`processing_time_ms`::: (object) The `avg`, `min`, `max` and `total` runtime in milliseconds for forecasts related to this job. If there are no forecasts, this property is omitted. `forecasts_stats`.`status`::: (object) The count of forecasts by their status. For example: {"finished" : 2, "started" : 1}. If there are no forecasts, this property is omitted. `forecasts_stats`.`total`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=forecast-total] `job_id`:: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] [[modelsizestats]]`model_size_stats`:: (object) An object that provides information about the size and contents of the model. It has the following properties: `model_size_stats`.`bucket_allocation_failures_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-allocation-failures-count] `model_size_stats`.`categorized_doc_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=categorized-doc-count] `model_size_stats`.`categorization_status`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-status] `model_size_stats`.`dead_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=dead-category-count] `model_size_stats`.`frequent_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=frequent-category-count] `model_size_stats`.`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] `model_size_stats`.`log_time`::: (date) The timestamp of the `model_size_stats` according to server time. `model_size_stats`.`memory_status`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-status] `model_size_stats`.`model_bytes`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=model-bytes] `model_size_stats`.`model_bytes_exceeded`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=model-bytes-exceeded] `model_size_stats`.`model_bytes_memory_limit`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit-anomaly-jobs] `model_size_stats`.`rare_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=rare-category-count] `model_size_stats`.`result_type`::: (string) For internal use. The type of result. `model_size_stats`.`total_by_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-by-field-count] `model_size_stats`.`total_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-category-count] `model_size_stats`.`total_over_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-over-field-count] `model_size_stats`.`total_partition_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-partition-field-count] `model_size_stats`.`timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=model-timestamp] [[stats-node]]`node`:: (object) Contains properties for the node that runs the job. This information is available only for open jobs. `node`.`attributes`::: (object) Lists node attributes. For example, `{"ml.machine_memory": "17179869184"}`. `node`.`ephemeral_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=node-ephemeral-id] `node`.`id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=node-id] `node`.`name`::: (string) The node name. `node`.`transport_address`::: (string) The host and port where transport HTTP connections are accepted. `open_time`:: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=open-time] `state`:: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=state-anomaly-job] [[timingstats]]`timing_stats`:: (object) An object that provides statistical information about timing aspect of this job. It has the following properties: `timing_stats`.`average_bucket_processing_time_ms`::: (double) Average of all bucket processing times in milliseconds. `timing_stats`.`bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-count] `timing_stats`.`exponential_average_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-exponential-average] `timing_stats`.`exponential_average_bucket_processing_time_per_hour_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-exponential-average-hour] `timing_stats`.`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] `timing_stats`.`maximum_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-maximum] `timing_stats`.`minimum_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-minimum] `timing_stats`.`total_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-total] [[ml-get-job-stats-response-codes]] ==== {api-response-codes-title} `404` (Missing resources):: If `allow_no_jobs` is `false`, this code indicates that there are no resources that match the request or only partial matches for the request. [[ml-get-job-stats-example]] ==== {api-examples-title} [source,console] -------------------------------------------------- GET _ml/anomaly_detectors/low_request_rate/_stats -------------------------------------------------- // TEST[skip:Kibana sample data] The API returns the following results: [source,js] ---- { "count" : 1, "jobs" : [ { "job_id" : "low_request_rate", "data_counts" : { "job_id" : "low_request_rate", "processed_record_count" : 1216, "processed_field_count" : 1216, "input_bytes" : 51678, "input_field_count" : 1216, "invalid_date_count" : 0, "missing_field_count" : 0, "out_of_order_timestamp_count" : 0, "empty_bucket_count" : 242, "sparse_bucket_count" : 0, "bucket_count" : 1457, "earliest_record_timestamp" : 1575172659612, "latest_record_timestamp" : 1580417369440, "last_data_time" : 1576017595046, "latest_empty_bucket_timestamp" : 1580356800000, "input_record_count" : 1216 }, "model_size_stats" : { "job_id" : "low_request_rate", "result_type" : "model_size_stats", "model_bytes" : 41480, "model_bytes_exceeded" : 0, "model_bytes_memory_limit" : 10485760, "total_by_field_count" : 3, "total_over_field_count" : 0, "total_partition_field_count" : 2, "bucket_allocation_failures_count" : 0, "memory_status" : "ok", "categorized_doc_count" : 0, "total_category_count" : 0, "frequent_category_count" : 0, "rare_category_count" : 0, "dead_category_count" : 0, "categorization_status" : "ok", "log_time" : 1576017596000, "timestamp" : 1580410800000 }, "forecasts_stats" : { "total" : 1, "forecasted_jobs" : 1, "memory_bytes" : { "total" : 9179.0, "min" : 9179.0, "avg" : 9179.0, "max" : 9179.0 }, "records" : { "total" : 168.0, "min" : 168.0, "avg" : 168.0, "max" : 168.0 }, "processing_time_ms" : { "total" : 40.0, "min" : 40.0, "avg" : 40.0, "max" : 40.0 }, "status" : { "finished" : 1 } }, "state" : "opened", "node" : { "id" : "7bmMXyWCRs-TuPfGJJ_yMw", "name" : "node-0", "ephemeral_id" : "hoXMLZB0RWKfR9UPPUCxXX", "transport_address" : "127.0.0.1:9300", "attributes" : { "ml.machine_memory" : "17179869184", "xpack.installed" : "true", "ml.max_open_jobs" : "20" } }, "assignment_explanation" : "", "open_time" : "13s", "timing_stats" : { "job_id" : "low_request_rate", "bucket_count" : 1457, "total_bucket_processing_time_ms" : 1094.000000000001, "minimum_bucket_processing_time_ms" : 0.0, "maximum_bucket_processing_time_ms" : 48.0, "average_bucket_processing_time_ms" : 0.75085792724777, "exponential_average_bucket_processing_time_ms" : 0.5571716855800993, "exponential_average_bucket_processing_time_per_hour_ms" : 15.0 } } ] } ----