From 0fa1060ca4329f9b02835b5d376e5948e18e3a63 Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Mon, 30 Mar 2020 11:06:33 -0700 Subject: [PATCH] [DOCS] Collapses content in machine learning APIs (#54234) (#54453) --- .../apis/estimate-model-memory.asciidoc | 2 +- .../apis/get-bucket.asciidoc | 32 ++-- .../apis/get-datafeed-stats.asciidoc | 37 ++-- .../apis/get-job-stats.asciidoc | 172 ++++++++++-------- .../apis/get-snapshot.asciidoc | 55 +++--- .../apis/post-calendar-event.asciidoc | 19 +- .../apis/put-datafeed.asciidoc | 1 + .../anomaly-detection/apis/put-job.asciidoc | 171 ++++++++++------- .../apis/update-datafeed.asciidoc | 1 + .../apis/update-job.asciidoc | 105 +++++++---- docs/reference/ml/ml-shared.asciidoc | 75 ++++---- .../apis/get-transform-stats.asciidoc | 120 +++++++++--- .../transform/apis/preview-transform.asciidoc | 93 +++++++--- .../transform/apis/put-transform.asciidoc | 62 +++++-- .../transform/apis/update-transform.asciidoc | 51 ++++-- 15 files changed, 645 insertions(+), 351 deletions(-) diff --git a/docs/reference/ml/anomaly-detection/apis/estimate-model-memory.asciidoc b/docs/reference/ml/anomaly-detection/apis/estimate-model-memory.asciidoc index 46a49f7cd23..ad615a2a8e9 100644 --- a/docs/reference/ml/anomaly-detection/apis/estimate-model-memory.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/estimate-model-memory.asciidoc @@ -26,7 +26,7 @@ configuration details and cardinality estimates for the fields it references. `analysis_config`:: (Required, object) For a list of the properties that you can specify in the -`analysis_config` component of the body of this API, see <>. +`analysis_config` component of the body of this API, see <>. `max_bucket_cardinality`:: (Optional, object) Estimates of the highest cardinality in a single bucket diff --git a/docs/reference/ml/anomaly-detection/apis/get-bucket.asciidoc b/docs/reference/ml/anomaly-detection/apis/get-bucket.asciidoc index 441c916304d..268d87591f6 100644 --- a/docs/reference/ml/anomaly-detection/apis/get-bucket.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/get-bucket.asciidoc @@ -77,6 +77,7 @@ default, the buckets are sorted by the `timestamp` field. `start`:: (Optional, string) Returns buckets with timestamps after this time. +[role="child_attributes"] [[ml-get-bucket-results]] ==== {api-response-body-title} @@ -89,49 +90,52 @@ records in the bucket contribute to this score. This value might be updated as new data is analyzed. `bucket_influencers`:: -(array) An array of bucket influencer objects, which have the following -properties: - -`bucket_influencers`.`anomaly_score`::: +(array) An array of bucket influencer objects. ++ +.Properties of `bucket_influencers` +[%collapsible%open] +==== +`anomaly_score`::: (number) A normalized score between 0-100, which is calculated for each bucket influencer. This score might be updated as newer data is analyzed. -`bucket_influencers`.`bucket_span`::: +`bucket_span`::: (number) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-span-results] -`bucket_influencers`.`initial_anomaly_score`::: +`initial_anomaly_score`::: (number) The score between 0-100 for each bucket influencer. This score is the initial value that was calculated at the time the bucket was processed. -`bucket_influencers`.`influencer_field_name`::: +`influencer_field_name`::: (string) The field name of the influencer. -`bucket_influencers`.`influencer_field_value`::: +`influencer_field_value`::: (string) The field value of the influencer. -`bucket_influencers`.`is_interim`::: +`is_interim`::: (boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=is-interim] -`bucket_influencers`.`job_id`::: +`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] -`bucket_influencers`.`probability`::: +`probability`::: (number) The probability that the bucket has this behavior, in the range 0 to 1. This value can be held to a high precision of over 300 decimal places, so the `anomaly_score` is provided as a human-readable and friendly interpretation of this. -`bucket_influencers`.`raw_anomaly_score`::: +`raw_anomaly_score`::: (number) Internal. -`bucket_influencers`.`result_type`::: +`result_type`::: (string) Internal. This value is always set to `bucket_influencer`. -`bucket_influencers`.`timestamp`::: +`timestamp`::: (date) The start time of the bucket for which these results were calculated. +==== `bucket_span`:: (number) diff --git a/docs/reference/ml/anomaly-detection/apis/get-datafeed-stats.asciidoc b/docs/reference/ml/anomaly-detection/apis/get-datafeed-stats.asciidoc index bf49722991a..e723e1dbc8f 100644 --- a/docs/reference/ml/anomaly-detection/apis/get-datafeed-stats.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/get-datafeed-stats.asciidoc @@ -60,6 +60,7 @@ all {dfeeds}. (Optional, boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=allow-no-datafeeds] +[role="child_attributes"] [[ml-get-datafeed-stats-results]] ==== {api-response-body-title} @@ -77,18 +78,23 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=datafeed-id] `node`:: (object) include::{docdir}/ml/ml-shared.asciidoc[tag=node-datafeeds] - -`node`.`id`::: ++ +-- +[%collapsible%open] +==== +`id`::: include::{docdir}/ml/ml-shared.asciidoc[tag=node-id] -`node`.`name`::: The node name. For example, `0-o0tOo`. +`name`::: The node name. For example, `0-o0tOo`. -`node`.`ephemeral_id`::: +`ephemeral_id`::: include::{docdir}/ml/ml-shared.asciidoc[tag=node-ephemeral-id] -`node`.`transport_address`::: The host and port where transport HTTP connections are +`transport_address`::: The host and port where transport HTTP connections are accepted. For example, `127.0.0.1:9300`. -`node`.`attributes`::: For example, `{"ml.machine_memory": "17179869184"}`. +`attributes`::: For example, `{"ml.machine_memory": "17179869184"}`. +==== +-- `state`:: (string) @@ -97,27 +103,32 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=state-datafeed] `timing_stats`:: (object) An object that provides statistical information about timing aspect of this {dfeed}. - -`timing_stats`.`average_search_time_per_bucket_ms`::: ++ +-- +[%collapsible%open] +==== +`average_search_time_per_bucket_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=search-bucket-avg] -`timing_stats`.`bucket_count`::: +`bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-count] -`timing_stats`.`exponential_average_search_time_per_hour_ms`::: +`exponential_average_search_time_per_hour_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=search-exp-avg-hour] -`timing_stats`.`job_id`::: +`job_id`::: include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] -`timing_stats`.`search_count`::: +`search_count`::: include::{docdir}/ml/ml-shared.asciidoc[tag=search-count] -`timing_stats`.`total_search_time_ms`::: +`total_search_time_ms`::: include::{docdir}/ml/ml-shared.asciidoc[tag=search-time] +==== +-- [[ml-get-datafeed-stats-response-codes]] diff --git a/docs/reference/ml/anomaly-detection/apis/get-job-stats.asciidoc b/docs/reference/ml/anomaly-detection/apis/get-job-stats.asciidoc index 334cb98e58d..1b8ff976862 100644 --- a/docs/reference/ml/anomaly-detection/apis/get-job-stats.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/get-job-stats.asciidoc @@ -50,6 +50,7 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection-default] (Optional, boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=allow-no-jobs] +[role="child_attributes"] [[ml-get-job-stats-results]] ==== {api-response-body-title} @@ -60,227 +61,244 @@ job: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=assignment-explanation-anomaly-jobs] +//Begin data_counts [[datacounts]]`data_counts`:: (object) An object that describes the quantity of input to the job and any related error counts. The `data_count` values are cumulative for the lifetime of a job. If a model snapshot is reverted or old results are deleted, the job counts are not reset. - -`data_counts`.`bucket_count`::: ++ +.Properties of `data_counts` +[%collapsible%open] +==== +`bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-count-anomaly-jobs] -`data_counts`.`earliest_record_timestamp`::: +`earliest_record_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=earliest-record-timestamp] -`data_counts`.`empty_bucket_count`::: +`empty_bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=empty-bucket-count] -`data_counts`.`input_bytes`::: +`input_bytes`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=input-bytes] -`data_counts`.`input_field_count`::: +`input_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=input-field-count] -`data_counts`.`input_record_count`::: +`input_record_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=input-record-count] -`data_counts`.`invalid_date_count`::: +`invalid_date_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=invalid-date-count] -`data_counts`.`job_id`::: +`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] -`data_counts`.`last_data_time`::: +`last_data_time`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=last-data-time] -`data_counts`.`latest_empty_bucket_timestamp`::: +`latest_empty_bucket_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=latest-empty-bucket-timestamp] -`data_counts`.`latest_record_timestamp`::: +`latest_record_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=latest-record-timestamp] -`data_counts`.`latest_sparse_bucket_timestamp`::: +`latest_sparse_bucket_timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=latest-sparse-record-timestamp] -`data_counts`.`missing_field_count`::: +`missing_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=missing-field-count] + The value of `processed_record_count` includes this count. -`data_counts`.`out_of_order_timestamp_count`::: +`out_of_order_timestamp_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=out-of-order-timestamp-count] -`data_counts`.`processed_field_count`::: +`processed_field_count`::: include::{docdir}/ml/ml-shared.asciidoc[tag=processed-field-count] -`data_counts`.`processed_record_count`::: +`processed_record_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=processed-record-count] -`data_counts`.`sparse_bucket_count`::: +`sparse_bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=sparse-bucket-count] +==== +//End data_counts `deleting`:: (boolean) Indicates that the process of deleting the job is in progress but not yet completed. It is only reported when `true`. +//Begin forecasts_stats [[forecastsstats]]`forecasts_stats`:: (object) An object that provides statistical information about forecasts belonging to this job. Some statistics are omitted if no forecasts have been -made. It has the following properties: +made. + --- -NOTE: `memory_bytes`, `records`, `processing_time_ms` and `status` require at -least one forecast. Otherwise, these fields are omitted. - --- - -`forecasts_stats`.`forecasted_jobs`::: +NOTE: Unless there is at least one forecast, `memory_bytes`, `records`, +`processing_time_ms` and `status` properties are omitted. ++ +.Properties of `forecasts_stats` +[%collapsible%open] +==== +`forecasted_jobs`::: (long) A value of `0` indicates that forecasts do not exist for this job. A value of `1` indicates that at least one forecast exists. -`forecasts_stats`.`memory_bytes`::: +`memory_bytes`::: (object) The `avg`, `min`, `max` and `total` memory usage in bytes for forecasts related to this job. If there are no forecasts, this property is omitted. -`forecasts_stats`.`processing_time_ms`::: -(object) Statistics about the forecast runtime in milliseconds: minimum, -maximum, average and total. +`processing_time_ms`::: +(object) The `avg`, `min`, `max` and `total` runtime in milliseconds for +forecasts related to this job. If there are no forecasts, this property is +omitted. -`forecasts_stats`.`records`::: +`records`::: (object) The `avg`, `min`, `max` and `total` number of `model_forecast` documents written for forecasts related to this job. If there are no forecasts, this property is omitted. -`forecasts_stats`.`processing_time_ms`::: -(object) The `avg`, `min`, `max` and `total` runtime in milliseconds for -forecasts related to this job. If there are no forecasts, this property is -omitted. - -`forecasts_stats`.`status`::: +`status`::: (object) The count of forecasts by their status. For example: {"finished" : 2, "started" : 1}. If there are no forecasts, this property is omitted. -`forecasts_stats`.`total`::: +`total`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=forecast-total] +==== +//End forecasts_stats `job_id`:: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] +//Begin model_size_stats [[modelsizestats]]`model_size_stats`:: (object) An object that provides information about the size and contents of the -model. It has the following properties: - -`model_size_stats`.`bucket_allocation_failures_count`::: +model. ++ +.Properties of `model_size_stats` +[%collapsible%open] +==== +`bucket_allocation_failures_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-allocation-failures-count] -`model_size_stats`.`categorized_doc_count`::: +`categorized_doc_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=categorized-doc-count] -`model_size_stats`.`categorization_status`::: +`categorization_status`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-status] -`model_size_stats`.`dead_category_count`::: +`dead_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=dead-category-count] -`model_size_stats`.`frequent_category_count`::: +`frequent_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=frequent-category-count] -`model_size_stats`.`job_id`::: +`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] -`model_size_stats`.`log_time`::: +`log_time`::: (date) The timestamp of the `model_size_stats` according to server time. -`model_size_stats`.`memory_status`::: +`memory_status`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-status] -`model_size_stats`.`model_bytes`::: +`model_bytes`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=model-bytes] -`model_size_stats`.`model_bytes_exceeded`::: +`model_bytes_exceeded`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=model-bytes-exceeded] -`model_size_stats`.`model_bytes_memory_limit`::: +`model_bytes_memory_limit`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit-anomaly-jobs] -`model_size_stats`.`rare_category_count`::: +`rare_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=rare-category-count] -`model_size_stats`.`result_type`::: +`result_type`::: (string) For internal use. The type of result. -`model_size_stats`.`total_by_field_count`::: +`total_by_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-by-field-count] -`model_size_stats`.`total_category_count`::: +`total_category_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-category-count] -`model_size_stats`.`total_over_field_count`::: +`total_over_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-over-field-count] -`model_size_stats`.`total_partition_field_count`::: +`total_partition_field_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=total-partition-field-count] -`model_size_stats`.`timestamp`::: +`timestamp`::: (date) include::{docdir}/ml/ml-shared.asciidoc[tag=model-timestamp] +==== +//End model_size_stats +//Begin node [[stats-node]]`node`:: (object) Contains properties for the node that runs the job. This information is available only for open jobs. - -`node`.`attributes`::: ++ +.Properties of `node` +[%collapsible%open] +==== +`attributes`::: (object) Lists node attributes. For example, `{"ml.machine_memory": "17179869184"}`. -`node`.`ephemeral_id`::: +`ephemeral_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=node-ephemeral-id] -`node`.`id`::: +`id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=node-id] -`node`.`name`::: +`name`::: (string) The node name. -`node`.`transport_address`::: +`transport_address`::: (string) The host and port where transport HTTP connections are accepted. +==== +//End node `open_time`:: (string) @@ -290,40 +308,46 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=open-time] (string) include::{docdir}/ml/ml-shared.asciidoc[tag=state-anomaly-job] +//Begin timing_stats [[timingstats]]`timing_stats`:: (object) An object that provides statistical information about timing aspect of -this job. It has the following properties: - -`timing_stats`.`average_bucket_processing_time_ms`::: +this job. ++ +.Properties of `timing_stats` +[%collapsible%open] +==== +`average_bucket_processing_time_ms`::: (double) Average of all bucket processing times in milliseconds. -`timing_stats`.`bucket_count`::: +`bucket_count`::: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-count] -`timing_stats`.`exponential_average_bucket_processing_time_ms`::: +`exponential_average_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-exponential-average] -`timing_stats`.`exponential_average_bucket_processing_time_per_hour_ms`::: +`exponential_average_bucket_processing_time_per_hour_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-exponential-average-hour] -`timing_stats`.`job_id`::: +`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] -`timing_stats`.`maximum_bucket_processing_time_ms`::: +`maximum_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-maximum] -`timing_stats`.`minimum_bucket_processing_time_ms`::: +`minimum_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-minimum] -`timing_stats`.`total_bucket_processing_time_ms`::: +`total_bucket_processing_time_ms`::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-total] +==== +//End timing_stats [[ml-get-job-stats-response-codes]] ==== {api-response-codes-title} diff --git a/docs/reference/ml/anomaly-detection/apis/get-snapshot.asciidoc b/docs/reference/ml/anomaly-detection/apis/get-snapshot.asciidoc index acf4467aad5..60ca9a3b20c 100644 --- a/docs/reference/ml/anomaly-detection/apis/get-snapshot.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/get-snapshot.asciidoc @@ -60,6 +60,7 @@ all model snapshots. `start`:: (Optional, string) Returns snapshots with timestamps after this time. +[role="child_attributes"] [[ml-get-snapshot-results]] ==== {api-response-body-title} @@ -82,17 +83,21 @@ the snapshot was created for. `min_version`:: (string) The minimum version required to be able to restore the model snapshot. +//Begin model_size_stats `model_size_stats`:: (object) Summary information describing the model. ++ +.Properties of `model_size_stats` +[%collapsible%open] +==== +`bucket_allocation_failures_count`::: +(long) The number of buckets for which entities were not processed due to memory +limit constraints. -`model_size_stats`.`bucket_allocation_failures_count`::: -(long) The number of buckets for which entities were not processed due to -memory limit constraints. - -`model_size_stats`.`categorized_doc_count`::: +`categorized_doc_count`::: (long) The number of documents that have had a field categorized. -`model_size_stats`.`categorization_status`::: +`categorization_status`::: (string) The status of categorization for this job. Contains one of the following values. + @@ -108,25 +113,25 @@ matched categories, or more than 50% of categories are dead. -- -`model_size_stats`.`dead_category_count`::: +`dead_category_count`::: (long) The number of categories created by categorization that will never be assigned again because another category's definition makes it a superset of the dead category. (Dead categories are a side effect of the way categorization has no prior training.) -`model_size_stats`.`frequent_category_count`::: +`frequent_category_count`::: (long) The number of categories that match more than 1% of categorized documents. -`model_size_stats`.`job_id`::: +`job_id`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] -`model_size_stats`.`log_time`::: +`log_time`::: (date) The timestamp that the `model_size_stats` were recorded, according to server-time. -`model_size_stats`.`memory_status`::: +`memory_status`::: (string) The status of the memory in relation to its `model_memory_limit`. Contains one of the following values. + @@ -139,38 +144,40 @@ memory limit and more aggressive pruning will be performed in order to try to reclaim space. -- -`model_size_stats`.`model_bytes`::: +`model_bytes`::: (long) An approximation of the memory resources required for this analysis. -`model_size_stats`.`model_bytes_exceeded`::: +`model_bytes_exceeded`::: (long) The number of bytes over the high limit for memory usage at the last allocation failure. -`model_size_stats`.`model_bytes_memory_limit`::: +`model_bytes_memory_limit`::: (long) The upper limit for memory usage, checked on increasing values. -`model_size_stats`.`rare_category_count`::: +`rare_category_count`::: (long) The number of categories that match just one categorized document. -`model_size_stats`.`result_type`::: -(string) Internal. This value is always set to `model_size_stats`. +`result_type`::: +(string) Internal. This value is always `model_size_stats`. -`model_size_stats`.`timestamp`::: -(date) The timestamp that the `model_size_stats` were recorded, according to -the bucket timestamp of the data. +`timestamp`::: +(date) The timestamp that the `model_size_stats` were recorded, according to the +bucket timestamp of the data. -`model_size_stats`.`total_by_field_count`::: +`total_by_field_count`::: (long) The number of _by_ field values analyzed. Note that these are counted separately for each detector and partition. -`model_size_stats`.`total_category_count`::: +`total_category_count`::: (long) The number of categories created by categorization. -`model_size_stats`.`total_over_field_count`::: +`total_over_field_count`::: (long) The number of _over_ field values analyzed. Note that these are counted separately for each detector and partition. -`model_size_stats`.`total_partition_field_count`::: +`total_partition_field_count`::: (long) The number of _partition_ field values analyzed. +==== +//End model_size_stats `retain`:: (boolean) diff --git a/docs/reference/ml/anomaly-detection/apis/post-calendar-event.asciidoc b/docs/reference/ml/anomaly-detection/apis/post-calendar-event.asciidoc index f981f24badc..246efc4e1bd 100644 --- a/docs/reference/ml/anomaly-detection/apis/post-calendar-event.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/post-calendar-event.asciidoc @@ -33,28 +33,29 @@ of which must have a start time, end time, and description. (Required, string) include::{docdir}/ml/ml-shared.asciidoc[tag=calendar-id] +[role="child_attributes"] [[ml-post-calendar-event-request-body]] ==== {api-request-body-title} `events`:: (Required, array) A list of one of more scheduled events. The event's start and end times may be specified as integer milliseconds since the epoch or as a -string in ISO 8601 format. An event resource has the following properties: - -`events`.`calendar_id`::: -(Optional, string) -include::{docdir}/ml/ml-shared.asciidoc[tag=calendar-id] - -`events`.`description`::: +string in ISO 8601 format. ++ +.Properties of events +[%collapsible%open] +==== +`description`::: (Optional, string) A description of the scheduled event. -`events`.`end_time`::: +`end_time`::: (Required, date) The timestamp for the end of the scheduled event in milliseconds since the epoch or ISO 8601 format. -`events`.`start_time`::: +`start_time`::: (Required, date) The timestamp for the beginning of the scheduled event in milliseconds since the epoch or ISO 8601 format. +==== [[ml-post-calendar-event-example]] ==== {api-examples-title} diff --git a/docs/reference/ml/anomaly-detection/apis/put-datafeed.asciidoc b/docs/reference/ml/anomaly-detection/apis/put-datafeed.asciidoc index 82f8d82bbfa..06e96e5ba92 100644 --- a/docs/reference/ml/anomaly-detection/apis/put-datafeed.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/put-datafeed.asciidoc @@ -51,6 +51,7 @@ those same roles. (Required, string) include::{docdir}/ml/ml-shared.asciidoc[tag=datafeed-id] +[role="child_attributes"] [[ml-put-datafeed-request-body]] ==== {api-request-body-title} diff --git a/docs/reference/ml/anomaly-detection/apis/put-job.asciidoc b/docs/reference/ml/anomaly-detection/apis/put-job.asciidoc index 216febc472a..cf8edae6fb5 100644 --- a/docs/reference/ml/anomaly-detection/apis/put-job.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/put-job.asciidoc @@ -35,6 +35,7 @@ a job directly to the `.ml-config` index using the {es} index API. If {es} (Required, string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection-define] +[role="child_attributes"] [[ml-put-job-request-body]] ==== {api-request-body-title} @@ -42,148 +43,172 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection-define] (Optional, boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=allow-lazy-open] +//Begin analysis_config [[put-analysisconfig]]`analysis_config`:: (Required, object) include::{docdir}/ml/ml-shared.asciidoc[tag=analysis-config] - -`analysis_config`.`bucket_span`::: ++ +.Properties of `analysis_config` +[%collapsible%open] +==== +`bucket_span`::: (<>) include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-span] -`analysis_config`.`categorization_field_name`::: -(string) -include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-field-name] - -`analysis_config`.`categorization_filters`::: -(array of strings) -include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-filters] - -`analysis_config`.`categorization_analyzer`::: +`categorization_analyzer`::: (object or string) include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-analyzer] -`analysis_config`.`detectors`::: +`categorization_field_name`::: +(string) +include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-field-name] + +`categorization_filters`::: +(array of strings) +include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-filters] + +//Begin analysis_config.detectors +`detectors`::: (array) An array of detector configuration objects. Detector configuration objects specify which data fields a job analyzes. They also specify which analytical functions are used. You can specify multiple detectors for a job. + --- NOTE: If the `detectors` array does not contain at least one detector, no analysis can occur and an error is returned. - -A detector has the following properties: --- - -`analysis_config`.`detectors`.`by_field_name`:::: ++ +.Properties of `detectors` +[%collapsible%open] +===== +`by_field_name`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=by-field-name] -[[put-customrules]]`analysis_config`.`detectors`.`custom_rules`:::: -+ --- +//Begin analysis_config.detectors.custom_rules +[[put-customrules]]`custom_rules`:::: (array) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules] ++ +.Properties of `custom_rules` +[%collapsible%open] +====== -`analysis_config`.`detectors`.`custom_rules`.`actions`::: +`actions`::: (array) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-actions] - -`analysis_config`.`detectors`.`custom_rules`.`scope`::: -(object) -include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope] - -`analysis_config`.`detectors`.`custom_rules`.`scope`.`filter_id`:::: -(string) -include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-id] -`analysis_config`.`detectors`.`custom_rules`.`scope`.`filter_type`:::: -(string) -include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-type] - -`analysis_config`.`detectors`.`custom_rules`.`conditions`::: +//Begin analysis_config.detectors.custom_rules.conditions +`conditions`::: (array) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions] - -`analysis_config`.`detectors`.`custom_rules`.`conditions`.`applies_to`:::: ++ +.Properties of `conditions` +[%collapsible%open] +======= +`applies_to`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions-applies-to] -`analysis_config`.`detectors`.`custom_rules`.`conditions`.`operator`:::: +`operator`:::: (string) -include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions-operator ] - -`analysis_config`.`detectors`.`custom_rules`.`conditions`.`value`:::: +include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions-operator] + +`value`:::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions-value] --- +======= +//End analysis_config.detectors.custom_rules.conditions -`analysis_config`.`detectors`.`detector_description`:::: +//Begin analysis_config.detectors.custom_rules.scope +`scope`::: +(object) +include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope] ++ +.Properties of `scope` +[%collapsible%open] +======= +`filter_id`:::: +(string) +include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-id] + +`filter_type`:::: +(string) +include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-type] +======= +//End analysis_config.detectors.custom_rules.scope +====== +//End analysis_config.detectors.custom_rules + +`detector_description`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=detector-description] -`analysis_config`.`detectors`.`detector_index`:::: -+ --- +`detector_index`:::: (integer) include::{docdir}/ml/ml-shared.asciidoc[tag=detector-index] ++ If you specify a value for this property, it is ignored. --- -`analysis_config`.`detectors`.`exclude_frequent`:::: +`exclude_frequent`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=exclude-frequent] -`analysis_config`.`detectors`.`field_name`:::: +`field_name`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=detector-field-name] -`analysis_config`.`detectors`.`function`:::: +`function`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=function] -`analysis_config`.`detectors`.`over_field_name`:::: +`over_field_name`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=over-field-name] -`analysis_config`.`detectors`.`partition_field_name`:::: +`partition_field_name`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=partition-field-name] -`analysis_config`.`detectors`.`use_null`:::: +`use_null`:::: (boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=use-null] +===== +//End analysis_config.detectors -`analysis_config`.`influencers`::: +`influencers`::: (array of strings) include::{docdir}/ml/ml-shared.asciidoc[tag=influencers] -`analysis_config`.`latency`::: +`latency`::: (<>) include::{docdir}/ml/ml-shared.asciidoc[tag=latency] -`analysis_config`.`multivariate_by_fields`::: +`multivariate_by_fields`::: (boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=multivariate-by-fields] -`analysis_config`.`summary_count_field_name`::: +`summary_count_field_name`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=summary-count-field-name] +==== +//End analysis_config +//Begin analysis_limits [[put-analysislimits]]`analysis_limits`:: (Optional, object) include::{docdir}/ml/ml-shared.asciidoc[tag=analysis-limits] + --- -The `analysis_limits` object has the following properties: --- - -`analysis_limits`.`categorization_examples_limit`::: -(long) +.Properties of `analysis_limits` +[%collapsible%open] +==== +`categorization_examples_limit`::: +(long) include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-examples-limit] - -`analysis_limits`.`model_memory_limit`::: -(long or string) + +`model_memory_limit`::: +(long or string) include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit] +==== +//End analysis_limits `background_persist_interval`:: (Optional, <>) @@ -193,9 +218,11 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=background-persist-interval] (Optional, object) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-settings] +//Begin data_description [[put-datadescription]]`data_description`:: (Required, object) include::{docdir}/ml/ml-shared.asciidoc[tag=data-description] +//End data_description `description`:: (Optional, string) A description of the job. @@ -204,17 +231,23 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=data-description] (Optional, array of strings) include::{docdir}/ml/ml-shared.asciidoc[tag=groups] +//Begin model_plot_config `model_plot_config`:: (Optional, object) include::{docdir}/ml/ml-shared.asciidoc[tag=model-plot-config] - -`model_plot_config`.`enabled`::: ++ +.Properties of `model_plot_config` +[%collapsible%open] +==== +`enabled`::: (boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=model-plot-config-enabled] -`model_plot_config`.`terms`::: +`terms`::: experimental[] (string) include::{docdir}/ml/ml-shared.asciidoc[tag=model-plot-config-terms] +==== +//End model_plot_config `model_snapshot_retention_days`:: (Optional, long) diff --git a/docs/reference/ml/anomaly-detection/apis/update-datafeed.asciidoc b/docs/reference/ml/anomaly-detection/apis/update-datafeed.asciidoc index 33c31589060..deb65df8e9c 100644 --- a/docs/reference/ml/anomaly-detection/apis/update-datafeed.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/update-datafeed.asciidoc @@ -42,6 +42,7 @@ using those same roles. (Required, string) include::{docdir}/ml/ml-shared.asciidoc[tag=datafeed-id] +[role="child_attributes"] [[ml-update-datafeed-request-body]] ==== {api-request-body-title} diff --git a/docs/reference/ml/anomaly-detection/apis/update-job.asciidoc b/docs/reference/ml/anomaly-detection/apis/update-job.asciidoc index aa7b6b87be4..8819451daf2 100644 --- a/docs/reference/ml/anomaly-detection/apis/update-job.asciidoc +++ b/docs/reference/ml/anomaly-detection/apis/update-job.asciidoc @@ -28,6 +28,7 @@ Updates certain properties of an {anomaly-job}. (Required, string) include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection] +[role="child_attributes"] [[ml-update-job-request-body]] ==== {api-request-body-title} @@ -42,8 +43,15 @@ NOTE: If the job is open when you make the update, you must stop the {dfeed}, close the job, then reopen the job and restart the {dfeed} for the changes to take effect. -- - -[[update-analysislimits]]`analysis_limits`.`model_memory_limit`:: +//Begin analysis_limits +[[update-analysislimits]]`analysis_limits`:: +(Optional, object) +include::{docdir}/ml/ml-shared.asciidoc[tag=analysis-limits] ++ +.Properties of `analysis_limits` +[%collapsible%open] +==== +`model_memory_limit`::: (long or string) include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit] + @@ -52,11 +60,13 @@ NOTE: You can update the `analysis_limits` only while the job is closed. The `model_memory_limit` property value cannot be decreased below the current usage. TIP: If the `memory_status` property in the -<> has a value of `hard_limit`, -this means that it was unable to process some data. You might want to re-run the -job with an increased `model_memory_limit`. +<> has a value of +`hard_limit`,mthis means that it was unable to process some data. You might want +to re-run the job with an increased `model_memory_limit`. -- +==== +//End analysis_limits `background_persist_interval`:: (<>) @@ -75,73 +85,104 @@ include::{docdir}/ml/ml-shared.asciidoc[tag=custom-settings] `description`:: (string) A description of the job. +//Begin detectors `detectors`:: (array) An array of detector update objects. - -`detectors`.`custom_rules`::: + --- +.Properties of `detectors` +[%collapsible%open] +==== + +//Begin detectors.custom_rules +`custom_rules`::: (array) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules] ++ +.Properties of `custom_rules` +[%collapsible%open] +===== -`detectors`.`custom_rules`.`actions`::: +`actions`::: (array) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-actions] -`detectors`.`custom_rules`.`scope`::: -(object) -include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope] - -`detectors`.`custom_rules`.`scope`.`filter_id`:::: -(string) -include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-id] - -`detectors`.`custom_rules`.`scope`.`filter_type`:::: -(string) -include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-type] - -`detectors`.`custom_rules`.`conditions`::: +// Begin detectors.custom_rules.conditions +`conditions`::: (array) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions] ++ +.Properties of `conditions` +[%collapsible%open] +====== -`detectors`.`custom_rules`.`conditions`.`applies_to`:::: +`applies_to`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions-applies-to] -`detectors`.`custom_rules`.`conditions`.`operator`:::: +`operator`:::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions-operator] -`detectors`.`custom_rules`.`conditions`.`value`:::: +`value`:::: (double) include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-conditions-value] --- +====== +//End detectors.custom_rules.conditions -`detectors`.`description`::: +//Begin detectors.custom_rules.scope +`scope`::: +(object) +include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope] ++ +.Properties of `scope` +[%collapsible%open] +====== +`filter_id`:::: +(string) +include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-id] + +`filter_type`:::: +(string) +include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules-scope-filter-type] +====== +//End detectors.custom_rules.scope +===== +//End detectors.custom_rules + +`description`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=detector-description] -`detectors`.`detector_index`::: -+ --- +`detector_index`::: (integer) include::{docdir}/ml/ml-shared.asciidoc[tag=detector-index] ++ +-- If you want to update a specific detector, you must use this identifier. You cannot, however, change the `detector_index` value for a detector. -- +==== +//End detectors `groups`:: (array of strings) include::{docdir}/ml/ml-shared.asciidoc[tag=groups] +//Begin model_plot_config `model_plot_config`:: (object) include::{docdir}/ml/ml-shared.asciidoc[tag=model-plot-config] - -`model_plot_config`.`enabled`::: ++ +.Properties of `model_plot_config` +[%collapsible%open] +==== +`enabled`::: (boolean) include::{docdir}/ml/ml-shared.asciidoc[tag=model-plot-config-enabled] +==== +//End model_plot_config + `model_snapshot_retention_days`:: (long) include::{docdir}/ml/ml-shared.asciidoc[tag=model-snapshot-retention-days] diff --git a/docs/reference/ml/ml-shared.asciidoc b/docs/reference/ml/ml-shared.asciidoc index 90955271cff..0be14751d11 100644 --- a/docs/reference/ml/ml-shared.asciidoc +++ b/docs/reference/ml/ml-shared.asciidoc @@ -88,7 +88,6 @@ tag::analysis-limits[] Limits can be applied for the resources required to hold the mathematical models in memory. These limits are approximate and can be set per job. They do not control the memory used by other processes, for example the {es} Java processes. -If necessary, you can increase the limits after the job is created. end::analysis-limits[] tag::analyzed-fields[] @@ -231,24 +230,26 @@ process. The syntax is very similar to that used to define the `analyzer` in the <>. For more information, see {ml-docs}/ml-configuring-categories.html[Categorizing log messages]. + --- The `categorization_analyzer` field can be specified either as a string or as an object. If it is a string it must refer to a <> or one added by another plugin. If it is an object it has the following properties: --- - -`analysis_config`.`categorization_analyzer`.`char_filter`:::: ++ +.Properties of `categorization_analyzer` +[%collapsible%open] +===== +`char_filter`:::: (array of strings or objects) include::{docdir}/ml/ml-shared.asciidoc[tag=char-filter] -`analysis_config`.`categorization_analyzer`.`tokenizer`:::: +`tokenizer`:::: (string or object) include::{docdir}/ml/ml-shared.asciidoc[tag=tokenizer] -`analysis_config`.`categorization_analyzer`.`filter`:::: +`filter`:::: (array of strings or objects) include::{docdir}/ml/ml-shared.asciidoc[tag=filter] +===== end::categorization-analyzer[] tag::categorization-examples-limit[] @@ -257,12 +258,9 @@ data store. The default value is `4`. If you increase this value, more examples are available, however it requires that you have more storage available. If you set this value to `0`, no examples are stored. + --- NOTE: The `categorization_examples_limit` only applies to analysis that uses categorization. For more information, see {ml-docs}/ml-configuring-categories.html[Categorizing log messages]. - --- end::categorization-examples-limit[] tag::categorization-field-name[] @@ -323,15 +321,18 @@ tag::chunking-config[] months or years. This search is split into time chunks in order to ensure the load on {es} is managed. Chunking configuration controls how the size of these time chunks are calculated and is an advanced configuration option. -A chunking configuration object has the following properties: - -`chunking_config`.`mode`::: ++ +.Properties of `chunking_config` +[%collapsible%open] +==== +`mode`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=mode] -`chunking_config`.`time_span`::: +`time_span`::: (<>) include::{docdir}/ml/ml-shared.asciidoc[tag=time-span] +==== end::chunking-config[] tag::class-assignment-objective[] @@ -419,23 +420,24 @@ end::custom-settings[] tag::data-description[] The data description defines the format of the input data when you send data to the job by using the <> API. Note that when configure -a {dfeed}, these properties are automatically set. +a {dfeed}, these properties are automatically set. When data is received via +the <> API, it is not stored in {es}. Only the results +for {anomaly-detect} are retained. + --- -When data is received via the <> API, it is not stored -in {es}. Only the results for {anomaly-detect} are retained. - -`data_description`.`format`::: +.Properties of `data_description` +[%collapsible%open] +==== +`format`::: (string) Only `JSON` format is supported at this time. -`data_description`.`time_field`::: +`time_field`::: (string) The name of the field that contains the timestamp. The default value is `time`. -`data_description`.`time_format`::: +`time_format`::: (string) include::{docdir}/ml/ml-shared.asciidoc[tag=time-format] --- +==== end::data-description[] tag::data-frame-analytics[] @@ -597,27 +599,29 @@ tag::delayed-data-check-config[] Specifies whether the {dfeed} checks for missing data and the size of the window. For example: `{"enabled": true, "check_window": "1h"}`. + --- The {dfeed} can optionally search over indices that have already been read in an effort to determine whether any data has subsequently been added to the index. If missing data is found, it is a good indication that the `query_delay` option is set too low and the data is being indexed after the {dfeed} has passed that moment in time. See {ml-docs}/ml-delayed-data-detection.html[Working with delayed data]. - ++ This check runs only on real-time {dfeeds}. - -`delayed_data_check_config`.`enabled`:: -(boolean) Specifies whether the {dfeed} periodically checks for delayed data. -Defaults to `true`. - -`delayed_data_check_config`.`check_window`:: ++ +.Properties of `delayed_data_check_config` +[%collapsible%open] +==== +`check_window`:: (<>) The window of time that is searched for late data. This window of time ends with the latest finalized bucket. It defaults to `null`, which causes an appropriate `check_window` to be calculated when the real-time {dfeed} runs. In particular, the default `check_window` span calculation is based on the maximum of `2h` or `8 * bucket_span`. --- + +`enabled`:: +(boolean) Specifies whether the {dfeed} periodically checks for delayed data. +Defaults to `true`. +==== end::delayed-data-check-config[] tag::dependent-variable[] @@ -1035,19 +1039,17 @@ cardinality fields, but the default is set to a relatively small size to ensure that high resource usage is a conscious decision. The default value for jobs created in versions earlier than 6.1 is `4096mb`. + --- If you specify a number instead of a string, the units are assumed to be MiB. Specifying a string is recommended for clarity. If you specify a byte size unit of `b` or `kb` and the number does not equate to a discrete number of megabytes, it is rounded down to the closest MiB. The minimum valid value is 1 MiB. If you specify a value less than 1 MiB, an error occurs. For more information about supported byte size units, see <>. - ++ If your `elasticsearch.yml` file contains an `xpack.ml.max_model_memory_limit` setting, an error occurs when you try to create jobs that have `model_memory_limit` values greater than that setting. For more information, see <>. --- end::model-memory-limit[] tag::model-memory-limit-anomaly-jobs[] @@ -1422,14 +1424,11 @@ seconds since 1 Jan 1970). The value `epoch_ms` indicates that time is measured in milliseconds since the epoch. The `epoch` and `epoch_ms` time formats accept either integer or real values. + + --- NOTE: Custom patterns must conform to the Java `DateTimeFormatter` class. When you use date-time formatting patterns, it is recommended that you provide the full date, time and time zone. For example: `yyyy-MM-dd'T'HH:mm:ssX`. If the pattern that you specify is not sufficient to produce a complete timestamp, job creation fails. - --- end::time-format[] tag::time-span[] diff --git a/docs/reference/transform/apis/get-transform-stats.asciidoc b/docs/reference/transform/apis/get-transform-stats.asciidoc index c528bfcc479..1d48709aa88 100644 --- a/docs/reference/transform/apis/get-transform-stats.asciidoc +++ b/docs/reference/transform/apis/get-transform-stats.asciidoc @@ -67,6 +67,7 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=from-transforms] (Optional, integer) include::{docdir}/rest-api/common-parms.asciidoc[tag=size-transforms] +[role="child_attributes"] [[get-transform-stats-response]] ==== {api-response-body-title} @@ -74,107 +75,168 @@ The API returns an array of statistics objects for {transforms}, which are sorted by the `id` value in ascending order. All of these properties are informational; you cannot update their values. +//Begin checkpointing `checkpointing`:: (object) Contains statistics about <>. -`checkpointing`.`changes_last_detected_at`::: ++ +.Properties of `checkpointing` +[%collapsible%open] +==== +`changes_last_detected_at`::: (date) include::{docdir}/rest-api/common-parms.asciidoc[tag=checkpointing-changes-last-detected-at] -`checkpointing`.`last`::: + +//Begin checkpointing.last +`last`::: (object) Contains statistics about the last completed checkpoint. -`checkpointing`.`last`.`checkpoint`:::: ++ +.Properties of `last` +[%collapsible%open] +===== +`checkpoint`:::: (integer) The sequence number for the checkpoint. -`checkpointing`.`last`.`time_upper_bound_millis`:::: +`time_upper_bound_millis`:::: (date) When using time-based synchronization, this timestamp indicates the upper bound of data that is included in the checkpoint. -`checkpointing`.`last`.`timestamp_millis`:::: +`timestamp_millis`:::: (date) The timestamp of the checkpoint, which indicates when the checkpoint was created. -`checkpointing`.`next`::: +===== +//End checkpointing.last + +//Begin checkpointing.next +`next`::: (object) Contains statistics about the next checkpoint that is currently in progress. This object appears only when the {transform} `state` is `indexing`. -`checkpointing`.`next`.`checkpoint`:::: ++ +.Properties of `next` +[%collapsible%open] +===== +`checkpoint`:::: (integer) The sequence number for the checkpoint. -`checkpointing`.`next`.`checkpoint_progress`:::: + +`checkpoint_progress`:::: (object) Contains statistics about the progress of the checkpoint. For example, it lists the `total_docs`, `docs_remaining`, `percent_complete`, `docs_processed`, and `docs_indexed`. This information is available only for batch {transforms} and the first checkpoint of {ctransforms}. -`checkpointing`.`next`.`time_upper_bound_millis`:::: + +`time_upper_bound_millis`:::: (date) When using time-based synchronization, this timestamp indicates the upper bound of data that is included in the checkpoint. -`checkpointing`.`next`.`timestamp_millis`:::: + +`timestamp_millis`:::: (date) The timestamp of the checkpoint, which indicates when the checkpoint was created. -`checkpointing`.`operations_behind`::: +===== +//End checkpointing.next + +`operations_behind`::: (integer) The number of operations that have occurred on the source index but have not been applied to the destination index yet. A high number can indicate that the {transform} is failing to keep up. +==== +//End checkpointing `id`:: (string) include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] +//Begin node `node`:: (object) For started {transforms} only, the node upon which the {transform} is started. -`node`.`attributes`::: ++ +.Properties of `node` +[%collapsible%open] +==== +`attributes`::: (object) A list of attributes for the node. -`node`.`ephemeral_id`::: + +`ephemeral_id`::: (string) The node ephemeral ID. -`node`.`id`::: + +`id`::: (string) The unique identifier of the node. For example, "0-o0tOoRTwKFZifatTWKNw". -`node`.`name`::: + +`name`::: (string) The node name. For example, `0-o0tOo`. -`node`.`transport_address`::: + +`transport_address`::: (string) The host and port where transport HTTP connections are accepted. For example, `127.0.0.1:9300`. +==== +//End node + `reason`:: (string) include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform-reason] + `state`:: (string) include::{docdir}/rest-api/common-parms.asciidoc[tag=state-transform] + +//Begin stats `stats`:: (object) An object that provides statistical information about the {transform}. -`stats`.`documents_indexed`::: ++ +.Properties of `stats` +[%collapsible%open] +==== + +`documents_indexed`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=docs-indexed] -`stats`.`documents_processed`::: + +`documents_processed`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=docs-processed] -`stats`.`exponential_avg_checkpoint_duration_ms`::: + +`exponential_avg_checkpoint_duration_ms`::: (double) include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-checkpoint-duration-ms] -`stats`.`exponential_avg_documents_indexed`::: + +`exponential_avg_documents_indexed`::: (double) include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-indexed] -`stats`.`exponential_avg_documents_processed`::: + +`exponential_avg_documents_processed`::: (double) include::{docdir}/rest-api/common-parms.asciidoc[tag=exponential-avg-documents-processed] -`stats`.`index_failures`::: + +`index_failures`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=index-failures] -`stats`.`index_time_in_ms`::: + +`index_time_in_ms`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=index-time-ms] -`stats`.`index_total`::: + +`index_total`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=index-total] -`stats`.`pages_processed`::: + +`pages_processed`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=pages-processed] -`stats`.`search_failures`::: + +`search_failures`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=search-failures] -`stats`.`search_time_in_ms`::: + +`search_time_in_ms`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=search-time-ms] -`stats`.`search_total`::: + +`search_total`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=search-total] -`stats`.`trigger_count`::: + +`trigger_count`::: (long) include::{docdir}/rest-api/common-parms.asciidoc[tag=trigger-count] +==== +//End stats [[get-transform-stats-response-codes]] ==== {api-response-codes-title} diff --git a/docs/reference/transform/apis/preview-transform.asciidoc b/docs/reference/transform/apis/preview-transform.asciidoc index 77effafa4ae..e06199169c1 100644 --- a/docs/reference/transform/apis/preview-transform.asciidoc +++ b/docs/reference/transform/apis/preview-transform.asciidoc @@ -42,6 +42,7 @@ might result in poor mappings. As a work-around, create the destination index or an index template with your preferred mappings before you start the {transform}. +[role="child_attributes"] [[preview-transform-request-body]] ==== {api-request-body-title} @@ -49,83 +50,127 @@ or an index template with your preferred mappings before you start the `description`:: (Optional, string) Free text description of the {transform}. +//Begin dest `dest`:: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest] - -`dest`.`index`::: ++ +.Properties of `dest` +[%collapsible%open] +==== +`index`::: (Optional, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index] -`dest`.`pipeline`::: +`pipeline`::: (Optional, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline] +==== +//End dest `frequency`:: (Optional, <>) include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency] +//Begin pivot `pivot`:: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot] ++ +.Properties of `pivot` +[%collapsible%open] +==== -`pivot`.`aggregations` or `aggs`::: +`aggregations` or `aggs`::: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-aggs] -`pivot`.`group_by`::: +`group_by`::: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-group-by] -`pivot`.`max_page_search_size`::: +`max_page_search_size`::: (Optional, integer) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-max-page-search-size] +==== +//End pivot +//Begin source `source`:: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-transforms] ++ +.Properties of `source` +[%collapsible%open] +==== -`source`.`index`::: +`index`::: (Required, string or array) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms] -`source`.`query`::: +`query`::: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-query-transforms] +==== +//End source +//Begin sync `sync`:: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync] - -`sync`.`time`::: ++ +.Properties of `sync` +[%collapsible%open] +==== +//Begin sync.time + +`time`::: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time] ++ +.Properties of `analysis_config` +[%collapsible%open] +===== -`sync`.`time`.`delay`:::: +`delay`:::: (Optional, <>) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-delay] -`sync`.`time`.`field`:::: +`field`:::: (Optional, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-field] +===== +//End sync.time +==== +//End sync - +[role="child_attributes"] [[preview-transform-response]] ==== {api-response-body-title} `preview`:: - (array) An array of documents. In particular, they are the JSON - representation of the documents that would be created in the destination index - by the {transform}. - +(array) An array of documents. In particular, they are the JSON representation +of the documents that would be created in the destination index by the +{transform}. + +//Begin generated_dest_index `generated_dest_index`:: - (object) Contains details about the destination index. - `mappings`::: - (object) The <> for each document in the destination index. - `settings`::: - (object) The <> for the destination - index. - `aliases`::: The aliases for the destination index. +(object) Contains details about the destination index. ++ +.Properties of `generated_dest_index` +[%collapsible%open] +==== + +`aliases`::: +(object) The aliases for the destination index. + +`mappings`::: +(object) The <> for each document in the destination index. + +`settings`::: +(object) The <> for the destination index. +==== +//End generated_dest_index ==== {api-examples-title} diff --git a/docs/reference/transform/apis/put-transform.asciidoc b/docs/reference/transform/apis/put-transform.asciidoc index 39f6483a6b8..4192b08e449 100644 --- a/docs/reference/transform/apis/put-transform.asciidoc +++ b/docs/reference/transform/apis/put-transform.asciidoc @@ -72,69 +72,101 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] behavior may be desired if the source index does not exist until after the {transform} is created. +[role="child_attributes"] [[put-transform-request-body]] ==== {api-request-body-title} `description`:: (Optional, string) Free text description of the {transform}. +//Begin dest `dest`:: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest] - -`dest`.`index`::: ++ +.Properties of `dest` +[%collapsible%open] +==== + +`index`::: (Required, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index] -`dest`.`pipeline`::: +`pipeline`::: (Optional, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline] +==== +//End dest `frequency`:: (Optional, <>) include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency] +//Begin pivot `pivot`:: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot] ++ +.Properties of `pivot` +[%collapsible%open] +==== -`pivot`.`aggregations` or `aggs`::: +`aggregations` or `aggs`::: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-aggs] -`pivot`.`group_by`::: +`group_by`::: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-group-by] -`pivot`.`max_page_search_size`::: +`max_page_search_size`::: (Optional, integer) include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-max-page-search-size] +==== +//End pivot +//Begin source `source`:: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-transforms] ++ +.Properties of `source` +[%collapsible%open] +==== -`source`.`index`::: +`index`::: (Required, string or array) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms] -`source`.`query`::: +`query`::: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-query-transforms] - +==== +//End source + +//Begin sync `sync`:: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync] - -`sync`.`time`::: ++ +.Properties of `sync` +[%collapsible%open] +==== + +//Begin time +`time`::: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time] ++ +.Properties of `time` +[%collapsible%open] +===== -`sync`.`time`.`delay`:::: +`delay`:::: (Optional, <>) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-delay] -`sync`.`time`.`field`:::: +`field`:::: (Required, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-field] + @@ -145,6 +177,10 @@ you might need to set the `delay` such that it accounts for data transmission delays. -- +===== +//End time +==== +//End sync [[put-transform-example]] ==== {api-examples-title} diff --git a/docs/reference/transform/apis/update-transform.asciidoc b/docs/reference/transform/apis/update-transform.asciidoc index 198fd477f57..afe7e9b549b 100644 --- a/docs/reference/transform/apis/update-transform.asciidoc +++ b/docs/reference/transform/apis/update-transform.asciidoc @@ -67,53 +67,78 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] behavior may be desired if the source index does not exist until after the {transform} is updated. +[role="child_attributes"] [[update-transform-request-body]] ==== {api-request-body-title} `description`:: (Optional, string) Free text description of the {transform}. +//Begin dest `dest`:: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest] - -`dest`.`index`::: ++ +.Properties of `dest` +[%collapsible%open] +==== + +`index`::: (Required, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index] -`dest`.`pipeline`::: +`pipeline`::: (Optional, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline] +==== +//End dest `frequency`:: (Optional, <>) include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency] +//Begin source `source`:: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-transforms] - -`source`.`index`::: ++ +.Properties of `source` +[%collapsible%open] +==== + +`index`::: (Required, string or array) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms] -`source`.`query`::: +`query`::: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=source-query-transforms] - +==== +//End source + +//Begin sync `sync`:: (Optional, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync] - -`sync`.`time`::: ++ +.Properties of `sync` +[%collapsible%open] +==== + +//Begin sync.time +`time`::: (Required, object) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time] ++ +.Properties of `time` +[%collapsible%open] +===== -`sync`.`time`.`delay`:::: +`delay`:::: (Optional, <>) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-delay] -`sync`.`time`.`field`:::: +`field`:::: (Required, string) include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-field] + @@ -124,6 +149,10 @@ you might need to set the `delay` such that it accounts for data transmission delays. -- +===== +//End sync.time +==== +//End sync [[update-transform-example]] ==== {api-examples-title}