diff --git a/docs/reference/docs/index_.asciidoc b/docs/reference/docs/index_.asciidoc index aa0715da8ac..c3d8ef502bf 100644 --- a/docs/reference/docs/index_.asciidoc +++ b/docs/reference/docs/index_.asciidoc @@ -180,7 +180,7 @@ automatic creation of indices that match specified patterns, or set it to comma-separated list of patterns you want to allow, or prefix each pattern with `+` or `-` to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow. -+ + IMPORTANT: The `action.auto_create_index` setting only affects the automatic creation of indices. It does not affect the creation of data streams. diff --git a/docs/reference/docs/reindex.asciidoc b/docs/reference/docs/reindex.asciidoc index e943943fca0..7f1dd14b895 100644 --- a/docs/reference/docs/reindex.asciidoc +++ b/docs/reference/docs/reindex.asciidoc @@ -7,7 +7,7 @@ Copies documents from a _source_ to a _destination_. The source and destination can be any pre-existing index, index alias, or -<>. However, the source and destination must be +<>. However, the source and destination must be different. For example, you cannot reindex a data stream into itself. [IMPORTANT] @@ -16,7 +16,8 @@ Reindex requires <> to be enabled for all documents in the source. The destination must exist and should be configured as wanted before calling `_reindex`. -Reindex does not copy the settings from the source or its associated template. +Reindex does not copy the settings from the source or its associated template. + Mappings, shard counts, replicas, and so on must be configured ahead of time. ================================================= @@ -70,8 +71,11 @@ POST _reindex [[docs-reindex-api-desc]] ==== {api-description-title} -Extracts the <> from the reindex request's source and indexes the documents into the destination. -You can copy all documents to the destination, or reindex a subset of the documents. +// tag::docs-reindex-api-desc-tag[] +Extracts the <> from the source index and indexes the documents into the destination index. +You can copy all documents to the destination index, or reindex a subset of the documents. +// end::docs-reindex-api-desc-tag[] + Just like <>, `_reindex` gets a snapshot of the source but its destination must be **different** so @@ -95,8 +99,8 @@ any reindex request to a destination data stream must have an `op_type` of`create`. A reindex can only add new documents to a destination data stream. It cannot update existing documents in a destination data stream. -By default, version conflicts abort the `_reindex` process. -To continue reindexing if there are conflicts, set the `"conflicts"` request body parameter to `proceed`. +By default, version conflicts abort the `_reindex` process. +To continue reindexing if there are conflicts, set the `"conflicts"` request body parameter to `proceed`. In this case, the response includes a count of the version conflicts that were encountered. Note that the handling of other error types is unaffected by the `"conflicts"` parameter. @@ -105,9 +109,9 @@ Note that the handling of other error types is unaffected by the `"conflicts"` p If the request contains `wait_for_completion=false`, {es} performs some preflight checks, launches the request, and returns a -<> you can use to cancel or get the status of the task. -{es} creates a record of this task as a document at `.tasks/task/${taskId}`. -When you are done with a task, you should delete the task document so +<> you can use to cancel or get the status of the task. +{es} creates a record of this task as a document at `.tasks/task/${taskId}`. +When you are done with a task, you should delete the task document so {es} can reclaim the space. [[docs-reindex-from-multiple-sources]] @@ -141,7 +145,7 @@ done Set `requests_per_second` to any positive decimal number (`1.4`, `6`, `1000`, etc.) to throttle the rate at which `_reindex` issues batches of index -operations. Requests are throttled by padding each batch with a wait time. +operations. Requests are throttled by padding each batch with a wait time. To disable throttling, set `requests_per_second` to `-1`. The throttling is done by waiting between batches so that the `scroll` that `_reindex` @@ -156,7 +160,7 @@ target_time = 1000 / 500 per second = 2 seconds wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds -------------------------------------------------- -Since the batch is issued as a single `_bulk` request, large batch sizes +Since the batch is issued as a single `_bulk` request, large batch sizes cause Elasticsearch to create many requests and then wait for a while before starting the next set. This is "bursty" instead of "smooth". @@ -447,13 +451,13 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=max_docs] ==== {api-request-body-title} `conflicts`:: -(Optional, enum) Set to `proceed` to continue reindexing even if there are conflicts. +(Optional, enum) Set to `proceed` to continue reindexing even if there are conflicts. Defaults to `abort`. `source`:: `index`::: -(Required, string) The name of the data stream, index, or index alias you are copying _from_. -Also accepts a comma-separated list to reindex from multiple sources. +(Required, string) The name of the data stream, index, or index alias you are copying _from_. +Also accepts a comma-separated list to reindex from multiple sources. `max_docs`::: (Optional, integer) The maximum number of documents to reindex. @@ -468,22 +472,22 @@ Required when indexing from remote. `username`:::: (Optional, string) The username to use for authentication with the remote host. `password`:::: -(Optional, string) The password to use for authentication with the remote host. -`socket_timeout`:::: +(Optional, string) The password to use for authentication with the remote host. +`socket_timeout`:::: (Optional, <>) The remote socket read timeout. Defaults to 30 seconds. -`connect_timeout`:::: +`connect_timeout`:::: (Optional, <>) The remote connection timeout. Defaults to 30 seconds. `size`::: -{Optional, integer) The number of documents to index per batch. -Use when indexing from remote to ensure that the batches fit within the on-heap buffer, -which defaults to a maximum size of 100 MB. +{Optional, integer) The number of documents to index per batch. +Use when indexing from remote to ensure that the batches fit within the on-heap buffer, +which defaults to a maximum size of 100 MB. `slice`::: `id`:::: -(Optional, integer) Slice ID for <>. +(Optional, integer) Slice ID for <>. `max`:::: -(Optional, integer) Total number of slices. +(Optional, integer) Total number of slices. `sort`::: + @@ -495,21 +499,21 @@ deprecated::[7.6, Sort in reindex is deprecated. Sorting in reindex was never gu -- `_source`::: -(Optional, string) If `true` reindexes all source fields. -Set to a list to reindex select fields. -Defaults to `true`. +(Optional, string) If `true` reindexes all source fields. +Set to a list to reindex select fields. +Defaults to `true`. `dest`:: `index`::: (Required, string) The name of the data stream, index, or index alias you are copying _to_. `version_type`::: -(Optional, enum) The versioning to use for the indexing operation. -Valid values: `internal`, `external`, `external_gt`, `external_gte`. +(Optional, enum) The versioning to use for the indexing operation. +Valid values: `internal`, `external`, `external_gt`, `external_gte`. See <> for more information. -`op_type`::: -(Optional, enum) Set to create to only index documents that do not already exist (put if absent). +`op_type`::: +(Optional, enum) Set to create to only index documents that do not already exist (put if absent). Valid values: `index`, `create`. Defaults to `index`. + IMPORTANT: To reindex to a data stream destination, this argument must be @@ -530,11 +534,10 @@ the special value `_doc`. See <> for further details. ==== `script`:: -`source`::: -(Optional, string) The script to run to update the document source or metadata when reindexing. - +`source`::: +(Optional, string) The script to run to update the document source or metadata when reindexing. `lang`::: -(Optional, enum) The script language: `painless`, `expression`, `mustache`, `java`. +(Optional, enum) The script language: `painless`, `expression`, `mustache`, `java`. For more information, see <>. @@ -638,7 +641,7 @@ POST _reindex [[docs-reindex-select-max-docs]] ===== Reindex select documents with `max_docs` -You can limit the number of processed documents by setting `max_docs`. +You can limit the number of processed documents by setting `max_docs`. For example, this request copies a single document from `twitter` to `new_twitter`: @@ -660,7 +663,7 @@ POST _reindex [[docs-reindex-multiple-sources]] ===== Reindex from multiple sources -The `index` attribute in `source` can be a list, allowing you to copy from lots +The `index` attribute in `source` can be a list, allowing you to copy from lots of sources in one request. This will copy documents from the `twitter` and `blog` indices: @@ -1047,7 +1050,7 @@ It is not possible to configure SSL in the body of the `_reindex` request. The following settings are supported: `reindex.ssl.certificate_authorities`:: -List of paths to PEM encoded certificate files that should be trusted. +List of paths to PEM encoded certificate files that should be trusted. You cannot specify both `reindex.ssl.certificate_authorities` and `reindex.ssl.truststore.path`. @@ -1072,7 +1075,7 @@ or "pkcs12", this setting defaults to `PKCS12`. Otherwise, it defaults to `jks`. `reindex.ssl.verification_mode`:: Indicates the type of verification to protect against man in the middle attacks -and certificate forgery. +and certificate forgery. One of `full` (verify the hostname and the certificate path), `certificate` (verify the certificate path, but not the hostname) or `none` (perform no verification - this is strongly discouraged in production environments). @@ -1092,12 +1095,12 @@ You cannot specify both `reindex.ssl.key` and `reindex.ssl.keystore.path`. `reindex.ssl.key_passphrase`:: Specifies the passphrase to decrypt the PEM encoded private key (`reindex.ssl.key`) if it is encrypted. -Cannot be used with `reindex.ssl.secure_key_passphrase`. +Cannot be used with `reindex.ssl.secure_key_passphrase`. `reindex.ssl.secure_key_passphrase` (<>):: Specifies the passphrase to decrypt the PEM encoded private key (`reindex.ssl.key`) if it is encrypted. -Cannot be used with `reindex.ssl.key_passphrase`. +Cannot be used with `reindex.ssl.key_passphrase`. `reindex.ssl.keystore.path`:: Specifies the path to the keystore that contains a private key and certificate @@ -1107,11 +1110,11 @@ You cannot specify both `reindex.ssl.key` and `reindex.ssl.keystore.path`. `reindex.ssl.keystore.type`:: The type of the keystore (`reindex.ssl.keystore.path`). Must be either `jks` or `PKCS12`. -If the keystore path ends in ".p12", ".pfx" or "pkcs12", this setting defaults +If the keystore path ends in ".p12", ".pfx" or "pkcs12", this setting defaults to `PKCS12`. Otherwise, it defaults to `jks`. `reindex.ssl.keystore.password`:: -The password to the keystore (`reindex.ssl.keystore.path`). This setting cannot be used +The password to the keystore (`reindex.ssl.keystore.path`). This setting cannot be used with `reindex.ssl.keystore.secure_password`. `reindex.ssl.keystore.secure_password` (<>):: @@ -1120,10 +1123,10 @@ This setting cannot be used with `reindex.ssl.keystore.password`. `reindex.ssl.keystore.key_password`:: The password for the key in the keystore (`reindex.ssl.keystore.path`). -Defaults to the keystore password. This setting cannot be used with +Defaults to the keystore password. This setting cannot be used with `reindex.ssl.keystore.secure_key_password`. `reindex.ssl.keystore.secure_key_password` (<>):: The password for the key in the keystore (`reindex.ssl.keystore.path`). -Defaults to the keystore password. This setting cannot be used with +Defaults to the keystore password. This setting cannot be used with `reindex.ssl.keystore.key_password`. diff --git a/docs/reference/index-modules.asciidoc b/docs/reference/index-modules.asciidoc index bc3da4745d1..a2f7c242170 100644 --- a/docs/reference/index-modules.asciidoc +++ b/docs/reference/index-modules.asciidoc @@ -12,6 +12,8 @@ an index. [[index-modules-settings]] == Index Settings +[[index-modules-settings-description]] +// tag::index-modules-settings-description-tag[] Index level settings can be set per-index. Settings may be: _static_:: @@ -23,6 +25,7 @@ _dynamic_:: They can be changed on a live index using the <> API. +// end::index-modules-settings-description-tag[] WARNING: Changing static or dynamic index settings on a closed index could result in incorrect settings that are impossible to rectify without deleting @@ -34,15 +37,14 @@ and recreating the index. Below is a list of all _static_ index settings that are not associated with any specific index module: -`index.number_of_shards`:: +[[index-number-of-shards]] +// tag::index-number-of-shards-tag[] +`index.number_of_shards` {ess-icon}:: +The number of primary shards that an index should have. Defaults to `1`. This setting can only be set at index creation time. It cannot be changed on a closed index. ++ +NOTE: The number of shards are limited to `1024` per index. This limitation is a safety limit to prevent accidental creation of indices that can destabilize a cluster due to resource allocation. The limit can be modified by specifying `export ES_JAVA_OPTS="-Des.index.max_number_of_shards=128"` system property on every node that is part of the cluster. - The number of primary shards that an index should have. Defaults to 1. - This setting can only be set at index creation time. It cannot be - changed on a closed index. Note: the number of shards are limited to `1024` per - index. This limitation is a safety limit to prevent accidental creation of indices - that can destabilize a cluster due to resource allocation. The limit can be modified - by specifying `export ES_JAVA_OPTS="-Des.index.max_number_of_shards=128"` system property on every node that is - part of the cluster. +// end::index-number-of-shards-tag[] `index.shard.check_on_startup`:: diff --git a/docs/reference/index-modules/blocks.asciidoc b/docs/reference/index-modules/blocks.asciidoc index b9cc29740ea..5709f62d428 100644 --- a/docs/reference/index-modules/blocks.asciidoc +++ b/docs/reference/index-modules/blocks.asciidoc @@ -22,6 +22,16 @@ index: Set to `true` to make the index and index metadata read only, `false` to allow writes and metadata changes. +`index.blocks.read_only_allow_delete`:: + + Similar to `index.blocks.read_only`, but also allows deleting the index to + make more resources available. The <> may add and remove this block automatically. ++ +Deleting documents from an index to release resources - rather than deleting the index itself - can increase the index size over time. When `index.blocks.read_only_allow_delete` is set to `true`, deleting documents is not permitted. However, deleting the index itself releases the read-only index block and makes resources available almost immediately. ++ +IMPORTANT: {es} adds and removes the read-only index block automatically when the disk utilization falls below the high watermark, controlled by <>. + `index.blocks.read`:: Set to `true` to disable read operations against the index. @@ -51,7 +61,7 @@ disk space. When you delete an index the data is removed from disk almost immediately, freeing the space it consumes. IMPORTANT: {es} adds the read-only-allow-delete index block automatically when -disk utilisation exceeds the <> and removes it again when disk utilisation is below the <>. You should not apply this block yourself. @@ -148,4 +158,3 @@ The API returns following response: } ] } -------------------------------------------------- - diff --git a/docs/reference/modules/cluster/disk_allocator.asciidoc b/docs/reference/modules/cluster/disk_allocator.asciidoc index 2acd72fc43f..21bca605686 100644 --- a/docs/reference/modules/cluster/disk_allocator.asciidoc +++ b/docs/reference/modules/cluster/disk_allocator.asciidoc @@ -1,6 +1,7 @@ [[disk-based-shard-allocation]] ==== Disk-based shard allocation settings - +[[disk-based-shard-allocation-description]] +// tag::disk-based-shard-allocation-description-tag[] {es} considers the available disk space on a node before deciding whether to allocate new shards to that node or to actively relocate shards away from that node. @@ -13,43 +14,42 @@ file or updated dynamically on a live cluster with the Defaults to `true`. Set to `false` to disable the disk allocation decider. -[[cluster-routing-watermark-low]] -`cluster.routing.allocation.disk.watermark.low`:: +[[cluster-routing-disk-threshold]] +// tag::cluster-routing-disk-threshold-tag[] +`cluster.routing.allocation.disk.threshold_enabled` {ess-icon}:: ++ +Defaults to `true`. Set to `false` to disable the disk allocation decider. +// end::cluster-routing-disk-threshold-tag[] - Controls the low watermark for disk usage. It defaults to `85%`, meaning - that {es} will not allocate shards to nodes that have more than - 85% disk used. It can also be set to an absolute byte value (like `500mb`) - to prevent {es} from allocating shards if less than the specified - amount of space is available. This setting has no effect on the primary - shards of newly-created indices but will prevent their replicas from being allocated. +[[cluster-routing-watermark-low]] +// tag::cluster-routing-watermark-low-tag[] +`cluster.routing.allocation.disk.watermark.low` {ess-icon}:: ++ +Controls the low watermark for disk usage. It defaults to `85%`, meaning that {es} will not allocate shards to nodes that have more than 85% disk used. It can also be set to an absolute byte value (like `500mb`) to prevent {es} from allocating shards if less than the specified amount of space is available. This setting has no effect on the primary shards of newly-created indices but will prevent their replicas from being allocated. +// end::cluster-routing-watermark-low-tag[] [[cluster-routing-watermark-high]] -`cluster.routing.allocation.disk.watermark.high`:: +// tag::cluster-routing-watermark-high-tag[] +`cluster.routing.allocation.disk.watermark.high` {ess-icon}:: ++ +Controls the high watermark. It defaults to `90%`, meaning that {es} will attempt to relocate shards away from a node whose disk usage is above 90%. It can also be set to an absolute byte value (similarly to the low watermark) to relocate shards away from a node if it has less than the specified amount of free space. This setting affects the allocation of all shards, whether previously allocated or not. +// end::cluster-routing-watermark-high-tag[] - Controls the high watermark. It defaults to `90%`, meaning that - {es} will attempt to relocate shards away from a node whose disk - usage is above 90%. It can also be set to an absolute byte value (similarly - to the low watermark) to relocate shards away from a node if it has less - than the specified amount of free space. This setting affects the - allocation of all shards, whether previously allocated or not. +`cluster.routing.allocation.disk.watermark.enable_for_single_data_node`:: + For a single data node, the default is to disregard disk watermarks when + making an allocation decision. This is deprecated behavior and will be + changed in 8.0. This setting can be set to `true` to enable the + disk watermarks for a single data node cluster (will become default in 8.0). -[[cluster-routing-flood_stage]] -`cluster.routing.allocation.disk.watermark.flood_stage`:: +[[cluster-routing-flood-stage]] +// tag::cluster-routing-flood-stage-tag[] +`cluster.routing.allocation.disk.watermark.flood_stage` {ess-icon}:: + -- -Controls the flood stage watermark, which defaults to 95%. {es} enforces a read-only index block -(`index.blocks.read_only_allow_delete`) on every index that has one or more -shards allocated on the node, and that has at least one disk exceeding the flood -stage. This setting is a last resort to prevent nodes from running out of disk space. -The index block is automatically released when the disk utilization falls below -the high watermark. +Controls the flood stage watermark, which defaults to 95%. {es} enforces a read-only index block (`index.blocks.read_only_allow_delete`) on every index that has one or more shards allocated on the node, and that has at least one disk exceeding the flood stage. This setting is a last resort to prevent nodes from running out of disk space. The index block is automatically released when the disk utilization falls below the high watermark. NOTE: You cannot mix the usage of percentage values and byte values within -these settings. Either all values are set to percentage values, or all are set to byte -values. This enforcement is so that {es} can validate that the settings are internally -consistent, ensuring that the low disk threshold is less than the high disk -threshold, and the high disk threshold is less than the flood stage -threshold. +these settings. Either all values are set to percentage values, or all are set to byte values. This enforcement is so that {es} can validate that the settings are internally consistent, ensuring that the low disk threshold is less than the high disk threshold, and the high disk threshold is less than the flood stage threshold. An example of resetting the read-only index block on the `twitter` index: @@ -62,6 +62,7 @@ PUT /twitter/_settings -------------------------------------------------- // TEST[setup:twitter] -- +// end::cluster-routing-flood-stage-tag[] `cluster.info.update.interval`:: diff --git a/docs/reference/modules/http.asciidoc b/docs/reference/modules/http.asciidoc index 5dac0df2b5c..db4df724780 100644 --- a/docs/reference/modules/http.asciidoc +++ b/docs/reference/modules/http.asciidoc @@ -1,6 +1,7 @@ [[modules-http]] === HTTP - +[[modules-http-description]] +// tag::modules-http-description-tag[] The HTTP layer exposes {es}'s REST APIs over HTTP. The HTTP mechanism is completely asynchronous in nature, meaning that @@ -13,105 +14,118 @@ http://en.wikipedia.org/wiki/Keepalive#HTTP_Keepalive[HTTP keep alive] when connecting for better performance and try to get your favorite client not to do http://en.wikipedia.org/wiki/Chunked_transfer_encoding[HTTP chunking]. +// end::modules-http-description-tag[] [http-settings] ==== HTTP settings -The settings in the table below can be configured for HTTP. Note that none of -them are dynamically updatable so for them to take effect they should be set in -the Elasticsearch <>. +The following settings can be configured for HTTP. These settings also use the common <>. -[cols="<,<",options="header",] -|======================================================================= -|Setting |Description -|`http.port` |A bind port range. Defaults to `9200-9300`. +NOTE: HTTP settings cannot be updated dynamically. You must configure these settings in the {es} <> +and restart {es} for changes to take effect. -|`http.publish_port` |The port that HTTP clients should use when +`http.port`:: +A bind port range. Defaults to `9200-9300`. + +`http.publish_port`:: +The port that HTTP clients should use when communicating with this node. Useful when a cluster node is behind a proxy or firewall and the `http.port` is not directly addressable from the outside. Defaults to the actual port assigned via `http.port`. -|`http.bind_host` |The host address to bind the HTTP service to. Defaults to `http.host` (if set) or `network.bind_host`. +`http.bind_host`:: +The host address to bind the HTTP service to. Defaults to `http.host` (if set) or `network.bind_host`. -|`http.publish_host` |The host address to publish for HTTP clients to connect to. Defaults to `http.host` (if set) or `network.publish_host`. +`http.publish_host`:: +The host address to publish for HTTP clients to connect to. Defaults to `http.host` (if set) or `network.publish_host`. -|`http.host` |Used to set the `http.bind_host` and the `http.publish_host`. +`http.host`:: +Used to set the `http.bind_host` and the `http.publish_host`. -|`http.max_content_length` |The max content of an HTTP request. Defaults to -`100mb`. +`http.max_content_length`:: +The max content of an HTTP request. Defaults to `100MB`. -|`http.max_initial_line_length` |The max length of an HTTP URL. Defaults -to `4kb` +`http.max_initial_line_length`:: +The max length of an HTTP URL. Defaults to `4KB`. -|`http.max_header_size` | The max size of allowed headers. Defaults to `8kB` - - -|`http.compression` |Support for compression when possible (with -Accept-Encoding). If HTTPS is enabled, defaults to `false`. Otherwise, defaults -to `true`. +`http.max_header_size`:: +The max size of allowed headers. Defaults to `8KB`. +[[http-compression]] +// tag::http-compression-tag[] +`http.compression` {ess-icon}:: +Support for compression when possible (with Accept-Encoding). If HTTPS is enabled, defaults to `false`. Otherwise, defaults to `true`. ++ Disabling compression for HTTPS mitigates potential security risks, such as a https://en.wikipedia.org/wiki/BREACH[BREACH attack]. To compress HTTPS traffic, you must explicitly set `http.compression` to `true`. +// end::http-compression-tag[] -|`http.compression_level` |Defines the compression level to use for HTTP responses. Valid values are in the range of 1 (minimum compression) -and 9 (maximum compression). Defaults to `3`. +`http.compression_level`:: +Defines the compression level to use for HTTP responses. Valid values are in the range of 1 (minimum compression) and 9 (maximum compression). Defaults to `3`. -|`http.cors.enabled` |Enable or disable cross-origin resource sharing, -i.e. whether a browser on another origin can execute requests against -Elasticsearch. Set to `true` to enable Elasticsearch to process pre-flight +[[http-cors-enabled]] +// tag::http-cors-enabled-tag[] +`http.cors.enabled` {ess-icon}:: +Enable or disable cross-origin resource sharing, which determines whether a browser on another origin can execute requests against {es}. Set to `true` to enable {es} to process pre-flight https://en.wikipedia.org/wiki/Cross-origin_resource_sharing[CORS] requests. -Elasticsearch will respond to those requests with the `Access-Control-Allow-Origin` header -if the `Origin` sent in the request is permitted by the `http.cors.allow-origin` -list. Set to `false` (the default) to make Elasticsearch ignore the `Origin` -request header, effectively disabling CORS requests because Elasticsearch will -never respond with the `Access-Control-Allow-Origin` response header. Note that -if the client does not send a pre-flight request with an `Origin` header or it -does not check the response headers from the server to validate the +{es} will respond to those requests with the `Access-Control-Allow-Origin` header if the `Origin` sent in the request is permitted by the `http.cors.allow-origin` list. Set to `false` (the default) to make {es} ignore the `Origin` request header, effectively disabling CORS requests because {es} will never respond with the `Access-Control-Allow-Origin` response header. ++ +NOTE: If the client does not send a pre-flight request with an `Origin` header or it does not check the response headers from the server to validate the `Access-Control-Allow-Origin` response header, then cross-origin security is -compromised. If CORS is not enabled on Elasticsearch, the only way for the client -to know is to send a pre-flight request and realize the required response headers -are missing. +compromised. If CORS is not enabled on {es}, the only way for the client to know is to send a pre-flight request and realize the required response headers are missing. -|`http.cors.allow-origin` |Which origins to allow. Defaults to no origins -allowed. If you prepend and append a `/` to the value, this will -be treated as a regular expression, allowing you to support HTTP and HTTPs. -for example using `/https?:\/\/localhost(:[0-9]+)?/` would return the -request header appropriately in both cases. `*` is a valid value but is -considered a *security risk* as your Elasticsearch instance is open to cross origin -requests from *anywhere*. +// end::http-cors-enabled-tag[] -|`http.cors.max-age` |Browsers send a "preflight" OPTIONS-request to -determine CORS settings. `max-age` defines how long the result should -be cached for. Defaults to `1728000` (20 days) +[[http-cors-allow-origin]] +// tag::http-cors-allow-origin-tag[] +`http.cors.allow-origin` {ess-icon}:: +Which origins to allow. If you prepend and append a forward slash (`/`) to the value, this will be treated as a regular expression, allowing you to support HTTP and HTTPs. For example, using `/https?:\/\/localhost(:[0-9]+)?/` would return the request header appropriately in both cases. Defaults to no origins allowed. ++ +IMPORTANT: A wildcard (`*`) is a valid value but is considered a security risk, as your {es} instance is open to cross origin requests from *anywhere*. -|`http.cors.allow-methods` |Which methods to allow. Defaults to -`OPTIONS, HEAD, GET, POST, PUT, DELETE`. +// end::http-cors-allow-origin-tag[] -|`http.cors.allow-headers` |Which headers to allow. Defaults to -`X-Requested-With, Content-Type, Content-Length`. +[[http-cors-max-age]] +// tag::http-cors-max-age-tag[] +`http.cors.max-age` {ess-icon}:: +Browsers send a "preflight" OPTIONS-request to determine CORS settings. `max-age` defines how long the result should be cached for. Defaults to `1728000` (20 days). +// end::http-cors-max-age-tag[] -|`http.cors.allow-credentials` | Whether the `Access-Control-Allow-Credentials` -header should be returned. Note: This header is only returned, when the setting is -set to `true`. Defaults to `false` +[[http-cors-allow-methods]] +// tag::http-cors-allow-methods-tag[] +`http.cors.allow-methods` {ess-icon}:: +Which methods to allow. Defaults to `OPTIONS, HEAD, GET, POST, PUT, DELETE`. +// end::http-cors-allow-methods-tag[] -|`http.detailed_errors.enabled` |Enables or disables the output of detailed error messages -and stack traces in response output. Note: When set to `false` and the `error_trace` request -parameter is specified, an error will be returned; when `error_trace` is not specified, a -simple message will be returned. Defaults to `true` +[[http-cors-allow-headers]] +// tag::http-cors-allow-headers-tag[] +`http.cors.allow-headers` {ess-icon}:: +Which headers to allow. Defaults to `X-Requested-With, Content-Type, Content-Length`. +// end::http-cors-allow-headers-tag[] -|`http.pipelining.max_events` |The maximum number of events to be queued up in memory before an HTTP connection is closed, defaults to `10000`. +[[http-cors-allow-credentials]] +// tag::http-cors-allow-credentials-tag[] +`http.cors.allow-credentials` {ess-icon}:: +Whether the `Access-Control-Allow-Credentials` header should be returned. Defaults to `false`. ++ +NOTE: This header is only returned when the setting is set to `true`. -|`http.max_warning_header_count` |The maximum number of warning headers in - client HTTP responses, defaults to unbounded. +// end::http-cors-allow-credentials-tag[] -|`http.max_warning_header_size` |The maximum total size of warning headers in -client HTTP responses, defaults to unbounded. +`http.detailed_errors.enabled`:: +If `true`, enables the output of detailed error messages and stack traces in the response output. Defaults to `true`. ++ +If `false`, use the `error_trace` parameter to <> and return detailed error messages. Otherwise, only a simple message will be returned. -|======================================================================= +`http.pipelining.max_events`:: +The maximum number of events to be queued up in memory before an HTTP connection is closed, defaults to `10000`. -It also uses the common -<>. +`http.max_warning_header_count`:: +The maximum number of warning headers in client HTTP responses. Defaults to `unbounded`. + +`http.max_warning_header_size`:: +The maximum total size of warning headers in client HTTP responses. Defaults to `unbounded`. [http-rest-request-tracer] ==== REST request tracer @@ -141,4 +155,4 @@ PUT _cluster/settings "http.tracer.exclude" : "" } } --------------------------------------------------- \ No newline at end of file +-------------------------------------------------- diff --git a/docs/reference/modules/indices/circuit_breaker.asciidoc b/docs/reference/modules/indices/circuit_breaker.asciidoc index 36fd5fc495f..406c193f2a6 100644 --- a/docs/reference/modules/indices/circuit_breaker.asciidoc +++ b/docs/reference/modules/indices/circuit_breaker.asciidoc @@ -1,13 +1,12 @@ [[circuit-breaker]] === Circuit breaker settings - -Elasticsearch contains multiple circuit breakers used to prevent operations from -causing an OutOfMemoryError. Each breaker specifies a limit for how much memory -it can use. Additionally, there is a parent-level breaker that specifies the -total amount of memory that can be used across all breakers. +[[circuit-breaker-description]] +// tag::circuit-breaker-description-tag[] +{es} contains multiple circuit breakers used to prevent operations from causing an OutOfMemoryError. Each breaker specifies a limit for how much memory it can use. Additionally, there is a parent-level breaker that specifies the total amount of memory that can be used across all breakers. Except where noted otherwise, these settings can be dynamically updated on a live cluster with the <> API. +// end::circuit-breaker-description-tag[] [[parent-circuit-breaker]] [float] @@ -21,11 +20,14 @@ The parent-level breaker can be configured with the following settings: memory usage into account (`true`) or only consider the amount that is reserved by child circuit breakers (`false`). Defaults to `true`. -`indices.breaker.total.limit`:: +[[indices-breaker-total-limit]] +// tag::indices-breaker-total-limit-tag[] +`indices.breaker.total.limit` {ess-icon}:: - Starting limit for overall parent breaker, defaults to 70% of JVM heap if + Starting limit for overall parent breaker. Defaults to 70% of JVM heap if `indices.breaker.total.use_real_memory` is `false`. If `indices.breaker.total.use_real_memory` is `true`, defaults to 95% of the JVM heap. +// end::indices-breaker-total-limit-tag[] [[fielddata-circuit-breaker]] [float] @@ -36,14 +38,20 @@ field data loading by raising an exception. By default the limit is configured to 40% of the maximum JVM heap. It can be configured with the following parameters: -`indices.breaker.fielddata.limit`:: +[[fielddata-circuit-breaker-limit]] +// tag::fielddata-circuit-breaker-limit-tag[] +`indices.breaker.fielddata.limit` {ess-icon}:: - Limit for fielddata breaker, defaults to 40% of JVM heap + Limit for fielddata breaker. Defaults to 40% of JVM heap. +// end::fielddata-circuit-breaker-limit-tag[] -`indices.breaker.fielddata.overhead`:: +[[fielddata-circuit-breaker-overhead]] +// tag::fielddata-circuit-breaker-overhead-tag[] +`indices.breaker.fielddata.overhead` {ess-icon}:: A constant that all field data estimations are multiplied with to determine a - final estimation. Defaults to 1.03 + final estimation. Defaults to `1.03`. +// end::fielddata-circuit-breaker-overhead-tag[] [[request-circuit-breaker]] [float] @@ -53,14 +61,20 @@ The request circuit breaker allows Elasticsearch to prevent per-request data structures (for example, memory used for calculating aggregations during a request) from exceeding a certain amount of memory. -`indices.breaker.request.limit`:: +[[request-breaker-limit]] +// tag::request-breaker-limit-tag[] +`indices.breaker.request.limit` {ess-icon}:: - Limit for request breaker, defaults to 60% of JVM heap + Limit for request breaker, defaults to 60% of JVM heap. +// end::request-breaker-limit-tag[] -`indices.breaker.request.overhead`:: +[[request-breaker-overhead]] +// tag::request-breaker-overhead-tag[] +`indices.breaker.request.overhead` {ess-icon}:: A constant that all request estimations are multiplied with to determine a - final estimation. Defaults to 1 + final estimation. Defaults to `1`. +// end::request-breaker-overhead-tag[] [[in-flight-circuit-breaker]] [float] diff --git a/docs/reference/modules/indices/index_management.asciidoc b/docs/reference/modules/indices/index_management.asciidoc new file mode 100644 index 00000000000..b9b4213e5a2 --- /dev/null +++ b/docs/reference/modules/indices/index_management.asciidoc @@ -0,0 +1,29 @@ +[[index-management-settings]] +=== Index management settings + +You can configure these index settings in the `elasticsearch.yml` file. + +[[auto-create-index]] +// tag::auto-create-index-tag[] +`action.auto_create_index` {ess-icon}:: +<> if it doesn't already exist and apply any configured index templates. Default: `true`. +// end::auto-create-index-tag[] + +[[action-destructive-requires-name]] +// tag::action-destructive-requires-name-tag[] +`action.destructive_requires_name` {ess-icon}:: +When set to `true`, you must specify the index name to <>. It is not possible to delete all indices with `_all` or use wildcards. +// end::action-destructive-requires-name-tag[] + +[[cluster-indices-close-enable]] +// tag::cluster-indices-close-enable-tag[] +`cluster.indices.close.enable` {ess-icon}:: +Enables <> in {es}. You might enable this setting temporarily to change the analyzer configuration for an existing index. We strongly recommend leaving this set to `false` (the default) otherwise. ++ +IMPORTANT: Closed indices are a data loss risk because they are not included when you make cluster configuration changes, such as scaling to a different capacity, failover, and many other operations. Additionally, closed indices can lead to inaccurate disk space counts. + +[[reindex-remote-whitelist]] +// tag::reindex-remote-whitelist[] +`reindex.remote.whitelist` {ess-icon}:: +Specifies the hosts that can be <>. Expects a YAML array of `host:port` strings. Consists of a comma-delimited list of `host:port` entries. Defaults to `["\*.io:*", "\*.com:*"]`. +// end::reindex-remote-whitelist[] diff --git a/docs/reference/modules/node.asciidoc b/docs/reference/modules/node.asciidoc index 01513a03e87..18717329bc5 100644 --- a/docs/reference/modules/node.asciidoc +++ b/docs/reference/modules/node.asciidoc @@ -9,13 +9,14 @@ Every node in the cluster can handle <> and <> traffic by default. The transport layer is used exclusively for communication between nodes; the HTTP layer is used by REST clients. - +[[modules-node-description]] +// tag::modules-node-description-tag[] All nodes know about all the other nodes in the cluster and can forward client requests to the appropriate node. By default, a node is all of the following types: master-eligible, data, ingest, and (if available) machine learning and transform. - +// end::modules-node-description-tag[] TIP: As the cluster grows and in particular if you have large {ml} jobs or {ctransforms}, consider separating dedicated master-eligible nodes from dedicated data nodes, {ml} nodes, and {transform} nodes. @@ -30,7 +31,7 @@ A node that has the `master` role (default), which makes it eligible to be A node that has the `data` role (default). Data nodes hold data and perform data related operations such as CRUD, search, and aggregations. -<>:: +<>:: A node that has the `ingest` role (default). Ingest nodes are able to apply an <> to a document in order to transform and enrich the @@ -199,6 +200,12 @@ To create a dedicated ingest node, set: node.roles: [ ingest ] ------------------- +[[node-ingest-node-setting]] +// tag::node-ingest-tag[] +`node.ingest` {ess-icon}:: +Determines whether a node is an ingest node. <> can apply an ingest pipeline to transform and enrich a document before indexing. Default: `true`. +// end::node-ingest-tag[] + [[coordinating-only-node]] ==== Coordinating only node diff --git a/docs/reference/settings/audit-settings.asciidoc b/docs/reference/settings/audit-settings.asciidoc index b4b3f6924b1..460724810da 100644 --- a/docs/reference/settings/audit-settings.asciidoc +++ b/docs/reference/settings/audit-settings.asciidoc @@ -4,17 +4,21 @@ ++++ Auditing settings ++++ - +[[auditing-settings-description]] +// tag::auditing-settings-description-tag[] You configure security auditing settings in the `elasticsearch.yml` configuration file on each node in the cluster. For more information, see <>. +// end::auditing-settings-description-tag[] [[general-audit-settings]] ==== General Auditing Settings - -`xpack.security.audit.enabled`:: +[[xpack-security-audit-enabled]] +// tag::xpack-security-audit-enabled-tag[] +`xpack.security.audit.enabled` {ess-icon}:: Set to `true` to enable auditing on the node. The default value is `false`. This puts the auditing events in a dedicated file named `_audit.json` -on each node. +on each node. +// end::xpack-security-audit-enabled-tag[] [[event-audit-settings]] ==== Audited Event Settings @@ -22,16 +26,24 @@ on each node. The events and some other information about what gets logged can be controlled by using the following settings: -`xpack.security.audit.logfile.events.include`:: +[[xpack-sa-lf-events-include]] +// tag::xpack-sa-lf-events-include-tag[] +`xpack.security.audit.logfile.events.include` {ess-icon}:: Specifies which events to include in the auditing output. The default value is: `access_denied, access_granted, anonymous_access_denied, authentication_failed, connection_denied, tampered_request, run_as_denied, run_as_granted`. +// end::xpack-sa-lf-events-include-tag[] -`xpack.security.audit.logfile.events.exclude`:: +[[xpack-sa-lf-events-exclude]] +// tag::xpack-sa-lf-events-exclude-tag[] +`xpack.security.audit.logfile.events.exclude` {ess-icon}:: Excludes the specified events from the output. By default, no events are excluded. +// end::xpack-sa-lf-events-exclude-tag[] -`xpack.security.audit.logfile.events.emit_request_body`:: +[[xpack-sa-lf-events-emit-request]] +// tag::xpack-sa-lf-events-emit-request-tag[] +`xpack.security.audit.logfile.events.emit_request_body` {ess-icon}:: Specifies whether to include the request body from REST requests on certain event types such as `authentication_failed`. The default value is `false`. + @@ -40,22 +52,35 @@ IMPORTANT: No filtering is performed when auditing, so sensitive data may be audited in plain text when including the request body in audit events. -- +// end::xpack-sa-lf-events-emit-request-tag[] + [[node-audit-settings]] ==== Local Node Info Settings -`xpack.security.audit.logfile.emit_node_name`:: +[[xpack-sa-lf-emit-node-name]] +// tag::xpack-sa-lf-emit-node-name-tag[] +`xpack.security.audit.logfile.emit_node_name` {ess-icon}:: Specifies whether to include the <> as a field in each audit event. The default value is `false`. +// end::xpack-sa-lf-emit-node-name-tag[] -`xpack.security.audit.logfile.emit_node_host_address`:: +[[xpack-sa-lf-emit-node-host-address]] +// tag::xpack-sa-lf-emit-node-host-address-tag[] +`xpack.security.audit.logfile.emit_node_host_address` {ess-icon}:: Specifies whether to include the node's IP address as a field in each audit event. The default value is `false`. +// end::xpack-sa-lf-emit-node-host-address-tag[] -`xpack.security.audit.logfile.emit_node_host_name`:: +[[xpack-sa-lf-emit-node-host-name]] +// tag::xpack-sa-lf-emit-node-host-name-tag[] +`xpack.security.audit.logfile.emit_node_host_name` {ess-icon}:: Specifies whether to include the node's host name as a field in each audit event. The default value is `false`. +// end::xpack-sa-lf-emit-node-host-name-tag[] -`xpack.security.audit.logfile.emit_node_id`:: +[[xpack-sa-lf-emit-node-id]] +// tag::xpack-sa-lf-emit-node-id-tag[] +`xpack.security.audit.logfile.emit_node_id` {ess-icon}:: Specifies whether to include the node id as a field in each audit event. This is available for the new format only. That is to say, this information does not exist in the `_access.log` file. @@ -63,6 +88,7 @@ Unlike <>, whose value might change if the administrator changes the setting in the config file, the node id will persist across cluster restarts and the administrator cannot change it. The default value is `true`. +// end::xpack-sa-lf-emit-node-id-tag[] [[audit-event-ignore-policies]] ==== Audit Logfile Event Ignore Policies @@ -70,25 +96,37 @@ The default value is `true`. These settings affect the <> that enable fine-grained control over which audit events are printed to the log file. All of the settings with the same policy name combine to form a single policy. -If an event matches all of the conditions for a specific policy, it is ignored +If an event matches all of the conditions for a specific policy, it is ignored and not printed. -`xpack.security.audit.logfile.events.ignore_filters..users`:: +[[xpack-sa-lf-events-ignore-users]] +// tag::xpack-sa-lf-events-ignore-users-tag[] +`xpack.security.audit.logfile.events.ignore_filters..users` {ess-icon}:: A list of user names or wildcards. The specified policy will not print audit events for users matching these values. +// end::xpack-sa-lf-events-ignore-users-tag[] -`xpack.security.audit.logfile.events.ignore_filters..realms`:: +[[xpack-sa-lf-events-ignore-realms]] +// tag::xpack-sa-lf-events-ignore-realms-tag[] +`xpack.security.audit.logfile.events.ignore_filters..realms` {ess-icon}:: A list of authentication realm names or wildcards. The specified policy will not print audit events for users in these realms. +// end::xpack-sa-lf-events-ignore-realms-tag[] -`xpack.security.audit.logfile.events.ignore_filters..roles`:: +[[xpack-sa-lf-events-ignore-roles]] +// tag::xpack-sa-lf-events-ignore-roles-tag[] +`xpack.security.audit.logfile.events.ignore_filters..roles` {ess-icon}:: A list of role names or wildcards. The specified policy will not print audit events for users that have these roles. If the user has several roles, some of which are *not* covered by the policy, the policy will *not* cover this event. +// end::xpack-sa-lf-events-ignore-roles-tag[] -`xpack.security.audit.logfile.events.ignore_filters..indices`:: +[[xpack-sa-lf-events-ignore-indices]] +// tag::xpack-sa-lf-events-ignore-indices-tag[] +`xpack.security.audit.logfile.events.ignore_filters..indices` {ess-icon}:: A list of index names or wildcards. The specified policy will not print audit events when all the indices in the event match these values. If the event concerns several indices, some of which are *not* covered by the policy, the policy will *not* cover this event. +// end::xpack-sa-lf-events-ignore-indices-tag[] diff --git a/docs/reference/settings/common-defs.asciidoc b/docs/reference/settings/common-defs.asciidoc index f434577c53a..ce66eb94b10 100644 --- a/docs/reference/settings/common-defs.asciidoc +++ b/docs/reference/settings/common-defs.asciidoc @@ -1,27 +1,44 @@ tag::ssl-certificate[] Specifies the path for the PEM encoded certificate (or certificate chain) that is associated with the key. -//TBD: This setting can be used only if `ssl.key` is set. ++ +This setting can be used only if `ssl.key` is set. end::ssl-certificate[] tag::ssl-certificate-authorities[] List of paths to PEM encoded certificate files that should be trusted. -//TBD: You cannot use this setting and `ssl.truststore.path` at the same time. ++ +This setting and `ssl.truststore.path` cannot be used at the same time. end::ssl-certificate-authorities[] tag::ssl-cipher-suites-values[] -include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-cipher-suites-values-java11] +Supported cipher suites vary depending on which version of Java you use. For +example, for version 12 the default value is `TLS_AES_256_GCM_SHA384`, +`TLS_AES_128_GCM_SHA256`, `TLS_CHACHA20_POLY1305_SHA256`, +`TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384`, `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`, +`TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384`, `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256`, +`TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256`, `TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256`, +`TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384`, `TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256`, +`TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384`, `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256`, +`TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA`, `TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA`, +`TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA`, `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA`, +`TLS_RSA_WITH_AES_256_GCM_SHA384`, `TLS_RSA_WITH_AES_128_GCM_SHA256`, +`TLS_RSA_WITH_AES_256_CBC_SHA256`, `TLS_RSA_WITH_AES_128_CBC_SHA256`, +`TLS_RSA_WITH_AES_256_CBC_SHA`, `TLS_RSA_WITH_AES_128_CBC_SHA`. ++ +For more information, see Oracle's +https://docs.oracle.com/en/java/javase/11/security/oracle-providers.html#GUID-7093246A-31A3-4304-AC5F-5FB6400405E2[Java Cryptography Architecture documentation]. end::ssl-cipher-suites-values[] tag::ssl-cipher-suites-values-java11[] Supported cipher suites vary depending on which version of Java you use. For -example, for version 11 the default value is `TLS_AES_256_GCM_SHA384`, -`TLS_AES_128_GCM_SHA256`, `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384`, -`TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`, `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384`, -`TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256`, `TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384`, -`TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256`, `TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384`, -`TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256`, `TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA`, -`TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA`, `TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA`, +example, for version 11 the default value is `TLS_AES_256_GCM_SHA384`, +`TLS_AES_128_GCM_SHA256`, `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384`, +`TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`, `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384`, +`TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256`, `TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384`, +`TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256`, `TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384`, +`TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256`, `TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA`, +`TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA`, `TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA`, `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA`, `TLS_RSA_WITH_AES_256_GCM_SHA384`, `TLS_RSA_WITH_AES_128_GCM_SHA256`, `TLS_RSA_WITH_AES_256_CBC_SHA256`, `TLS_RSA_WITH_AES_128_CBC_SHA256`, `TLS_RSA_WITH_AES_256_CBC_SHA`, @@ -44,17 +61,22 @@ end::ssl-cipher-suites-values-java11[] tag::ssl-key-pem[] Path to a PEM encoded file containing the private key. -//TBD: You cannot use this setting and `ssl.keystore.path` at the same time. ++ +If HTTP client authentication is required, it uses this file. You cannot use +this setting and `ssl.keystore.path` at the same time. end::ssl-key-pem[] tag::ssl-key-passphrase[] The passphrase that is used to decrypt the private key. Since the key might not be encrypted, this value is optional. -//TBD: You cannot use this setting and `ssl.secure_key_passphrase` at the same time. ++ +You cannot use this setting and `ssl.secure_key_passphrase` at the same time. end::ssl-key-passphrase[] tag::ssl-keystore-key-password[] The password for the key in the keystore. The default is the keystore password. ++ +You cannot use this setting and `ssl.keystore.secure_password` at the same time. //TBD: You cannot use this setting and `ssl.keystore.secure_key_password` at the same time. end::ssl-keystore-key-password[] @@ -65,6 +87,9 @@ end::ssl-keystore-password[] tag::ssl-keystore-path[] The path for the keystore file that contains a private key and certificate. ++ +It must be either a Java keystore (jks) or a PKCS#12 file. You cannot use this +setting and `ssl.key` at the same time. //TBD: It must be either a Java keystore (jks) or a PKCS#12 file. //TBD: You cannot use this setting and `ssl.key` at the same time. end::ssl-keystore-path[] @@ -81,13 +106,13 @@ end::ssl-keystore-secure-password[] tag::ssl-keystore-type-pkcs12[] The format of the keystore file. It must be either `jks` or `PKCS12`. If the -keystore path ends in ".p12", ".pfx", or ".pkcs12", this setting defaults +keystore path ends in ".p12", ".pfx", or ".pkcs12", this setting defaults to `PKCS12`. Otherwise, it defaults to `jks`. end::ssl-keystore-type-pkcs12[] tag::ssl-secure-key-passphrase[] The passphrase that is used to decrypt the private key. Since the key might not -be encrypted, this value is optional. +be encrypted, this value is optional. //TBD: You cannot use this setting and `ssl.key_passphrase` at the same time. end::ssl-secure-key-passphrase[] @@ -98,7 +123,7 @@ the default is `TLSv1.3,TLSv1.2,TLSv1.1`. Otherwise, the default is `TLSv1.2,TLSv1.1`. + -- -NOTE: If `xpack.security.fips_mode.enabled` is `true`, you cannot use `SSLv2Hello` +NOTE: If `xpack.security.fips_mode.enabled` is `true`, you cannot use `SSLv2Hello` or `SSLv3`. See <>. -- @@ -106,12 +131,17 @@ end::ssl-supported-protocols[] tag::ssl-truststore-password[] The password for the truststore. ++ +You cannot use this setting and `ssl.truststore.secure_password` at the same +time. //TBD: You cannot use this setting and `ssl.truststore.secure_password` at the same time. end::ssl-truststore-password[] tag::ssl-truststore-path[] The path for the keystore that contains the certificates to trust. It must be either a Java keystore (jks) or a PKCS#12 file. ++ +You cannot use this setting and `ssl.certificate_authorities` at the same time. //TBD: You cannot use this setting and `ssl.certificate_authorities` at the same time. end::ssl-truststore-path[] @@ -133,13 +163,16 @@ PKCS#12 files, use `PKCS12`. For a PKCS#11 token, use `PKCS11`. The default is end::ssl-truststore-type-pkcs11[] tag::ssl-verification-mode-values[] +Controls the verification of certificates. ++ Valid values are: -- `full`, which verifies that the provided certificate is signed by a trusted + + * `full`, which verifies that the provided certificate is signed by a trusted authority (CA) and also verifies that the server's hostname (or IP address) matches the names identified within the certificate. -- `certificate`, which verifies that the provided certificate is signed by a + * `certificate`, which verifies that the provided certificate is signed by a trusted authority (CA), but does not perform any hostname verification. -- `none`, which performs _no verification_ of the server's certificate. This + * `none`, which performs _no verification_ of the server's certificate. This mode disables many of the security benefits of SSL/TLS and should only be used after very careful consideration. It is primarily intended as a temporary diagnostic mechanism when attempting to resolve TLS errors; its use on diff --git a/docs/reference/settings/ml-settings.asciidoc b/docs/reference/settings/ml-settings.asciidoc index 73bfcd36463..aea9a9919ab 100644 --- a/docs/reference/settings/ml-settings.asciidoc +++ b/docs/reference/settings/ml-settings.asciidoc @@ -6,6 +6,8 @@ Machine learning settings ++++ +[[ml-settings-description]] +// tag::ml-settings-description-tag[] You do not need to configure any settings to use {ml}. It is enabled by default. IMPORTANT: {ml-cap} uses SSE4.2 instructions, so will only work on machines whose @@ -13,13 +15,15 @@ CPUs https://en.wikipedia.org/wiki/SSE4#Supporting_CPUs[support] SSE4.2. If you run {es} on older hardware you must disable {ml} (by setting `xpack.ml.enabled` to `false`). -All of these settings can be added to the `elasticsearch.yml` configuration file. -The dynamic settings can also be updated across a cluster with the +All of these settings can be added to the `elasticsearch.yml` configuration file. +The dynamic settings can also be updated across a cluster with the <>. -TIP: Dynamic settings take precedence over settings in the `elasticsearch.yml` +TIP: Dynamic settings take precedence over settings in the `elasticsearch.yml` file. +// end::ml-settings-description-tag[] + [float] [[general-ml-settings]] ==== General machine learning settings @@ -58,13 +62,16 @@ heap on each ingest node. The cache affords faster processing times for the or a percentage of total allocated heap. The default is "40%". See also <>. -`xpack.ml.inference_model.time_to_live`:: +[[xpack-interference-model-ttl]] +// tag::interference-model-ttl-tag[] +`xpack.ml.inference_model.time_to_live` {ess-icon}:: The time to live (TTL) for models in the inference model cache. The TTL is calculated from last access. The `inference` processor attempts to load the model from cache. If the `inference` processor does not receive any documents for the duration of the TTL, the referenced model is flagged for eviction from the cache. If a document is processed later, the model is again loaded into the cache. Defaults to `5m`. +// end::interference-model-ttl-tag[] `xpack.ml.max_inference_processors` (<>):: The total number of `inference` type processors allowed across all ingest @@ -88,12 +95,12 @@ affected when you update this setting. For more information about the [[xpack.ml.max_open_jobs]] `xpack.ml.max_open_jobs` (<>):: The maximum number of jobs that can run simultaneously on a node. Defaults to -`20`. In this context, jobs include both {anomaly-jobs} and {dfanalytics-jobs}. -The maximum number of jobs is also constrained by memory usage. Thus if the -estimated memory usage of the jobs would be higher than allowed, fewer jobs will -run on a node. Prior to version 7.1, this setting was a per-node non-dynamic -setting. It became a cluster-wide dynamic setting in version 7.1. As a result, -changes to its value after node startup are used only after every node in the +`20`. In this context, jobs include both {anomaly-jobs} and {dfanalytics-jobs}. +The maximum number of jobs is also constrained by memory usage. Thus if the +estimated memory usage of the jobs would be higher than allowed, fewer jobs will +run on a node. Prior to version 7.1, this setting was a per-node non-dynamic +setting. It became a cluster-wide dynamic setting in version 7.1. As a result, +changes to its value after node startup are used only after every node in the cluster is running version 7.1 or higher. The maximum permitted value is `512`. `xpack.ml.node_concurrent_job_allocations` (<>):: @@ -106,14 +113,14 @@ opening spend more time in the `opening` state. Defaults to `2`. [[advanced-ml-settings]] ==== Advanced machine learning settings -These settings are for advanced use cases; the default values are generally +These settings are for advanced use cases; the default values are generally sufficient: `xpack.ml.enable_config_migration` (<>):: Reserved. `xpack.ml.max_anomaly_records` (<>):: -The maximum number of records that are output per bucket. The default value is +The maximum number of records that are output per bucket. The default value is `500`. `xpack.ml.max_lazy_ml_nodes` (<>):: diff --git a/docs/reference/settings/monitoring-settings.asciidoc b/docs/reference/settings/monitoring-settings.asciidoc index bf567bf3b98..f142c5b4146 100644 --- a/docs/reference/settings/monitoring-settings.asciidoc +++ b/docs/reference/settings/monitoring-settings.asciidoc @@ -34,9 +34,12 @@ This deprecated setting has no effect. [[monitoring-collection-settings]] ==== Monitoring Collection Settings +[[monitoring-settings-description]] +// tag::monitoring-settings-description-tag[] The `xpack.monitoring.collection` settings control how data is collected from your Elasticsearch nodes. You can dynamically change all monitoring collection settings using the <>. +// end::monitoring-settings-description-tag[] `xpack.monitoring.collection.enabled` (<>):: @@ -45,7 +48,9 @@ this setting is `false` (default), {es} monitoring data is not collected and all monitoring data from other sources such as {kib}, Beats, and Logstash is ignored. -`xpack.monitoring.collection.interval` (<>):: +[[xpack-monitoring-collection-interval]] +// tag::monitoring-collection-interval-tag[] +`xpack.monitoring.collection.interval` (<>) {ess-icon}:: Setting to `-1` to disable data collection is no longer supported beginning with 7.0.0. deprecated[6.3.0, Use `xpack.monitoring.collection.enabled` set to `false` instead.] @@ -53,6 +58,7 @@ Setting to `-1` to disable data collection is no longer supported beginning with Controls how often data samples are collected. Defaults to `10s`. If you modify the collection interval, set the `xpack.monitoring.min_interval_seconds` option in `kibana.yml` to the same value. +// end::monitoring-collection-interval-tag[] `xpack.monitoring.elasticsearch.collection.enabled` (<>):: @@ -92,7 +98,9 @@ collect only active recoveries. Defaults to `false`. (<>) Timeout for collecting the recovery information. Defaults to `10s`. -`xpack.monitoring.history.duration` (<>):: +[[xpack-monitoring-history-duration]] +// tag::monitoring-history-duration-tag[] +`xpack.monitoring.history.duration` (<>) {ess-icon}:: (<>) Retention duration beyond which the indices created by a Monitoring exporter are automatically deleted. Defaults to `7d` (7 days). @@ -105,6 +113,8 @@ IMPORTANT: This setting currently only impacts `local`-type exporters. Indices c the `http` exporter will not be deleted automatically. -- +// end::monitoring-history-duration-tag[] + `xpack.monitoring.exporters`:: Configures where the agent stores monitoring data. By default, the agent uses a diff --git a/docs/reference/settings/notification-settings.asciidoc b/docs/reference/settings/notification-settings.asciidoc index 1bf4362a4e8..5661e5ed31f 100644 --- a/docs/reference/settings/notification-settings.asciidoc +++ b/docs/reference/settings/notification-settings.asciidoc @@ -7,6 +7,8 @@ {watcher} settings ++++ +[[notification-settings-description]] +// tag::notification-settings-description-tag[] You configure {watcher} settings to set up {watcher} and send notifications via <>, <>, and @@ -17,6 +19,7 @@ with the exception of the secure settings, which you add to the {es} keystore. For more information about creating and updating the {es} keystore, see <>. Dynamic settings can also be updated across a cluster with the <>. +// end::notification-settings-description-tag[] [float] [[general-notification-settings]] @@ -24,17 +27,22 @@ For more information about creating and updating the {es} keystore, see `xpack.watcher.enabled`:: Set to `false` to disable {watcher} on the node. -`xpack.watcher.encrypt_sensitive_data`:: +[[xpack-watcher-encrypt-sensitive-data]] +// tag::watcher-encrypt-sensitive-data-tag[] +`xpack.watcher.encrypt_sensitive_data` {ess-icon}:: Set to `true` to encrypt sensitive data. If this setting is enabled, you must also specify the `xpack.watcher.encryption_key` setting. For more information, see <>. +// end::watcher-encrypt-sensitive-data-tag[] `xpack.watcher.encryption_key` (<>):: Specifies the path to a file that contains a key for encrypting sensitive data. If `xpack.watcher.encrypt_sensitive_data` is set to `true`, this setting is required. For more information, see <>. -`xpack.watcher.history.cleaner_service.enabled`:: +[[xpack-watcher-history-cleaner-service]] +// tag::watcher-history-cleaner-service-tag[] +`xpack.watcher.history.cleaner_service.enabled` {ess-icon}:: added:[6.3.0,Default changed to `true`.] deprecated:[7.0.0,Watcher history indices are now managed by the `watch-history-ilm-policy` ILM policy] + @@ -44,6 +52,7 @@ removes previous versions of {watcher} indices (for example, {watcher} indices is determined by the `xpack.monitoring.history.duration` setting, which defaults to 7 days. For more information about that setting, see <>. +// end::watcher-history-cleaner-service-tag[] `xpack.http.proxy.host`:: Specifies the address of the proxy server to use to connect to HTTP services. diff --git a/docs/reference/settings/security-settings.asciidoc b/docs/reference/settings/security-settings.asciidoc index 28bce859238..d8b25cea612 100644 --- a/docs/reference/settings/security-settings.asciidoc +++ b/docs/reference/settings/security-settings.asciidoc @@ -36,8 +36,8 @@ those `kibana.yml` files. For more information about disabling {security-feature in specific {kib} instances, see {kibana-ref}/security-settings-kb.html[{kib} security settings]. -TIP: If you have gold or higher licenses, the default value is `true`; we -recommend that you explicitly add this setting to avoid confusion. +TIP: If you have gold or higher licenses, the default value is `true`; we +recommend that you explicitly add this setting to avoid confusion. -- @@ -57,7 +57,7 @@ Enables fips mode of operation. Set this to `true` if you run this {es} instance [[password-hashing-settings]] ==== Password hashing settings `xpack.security.authc.password_hashing.algorithm`:: -Specifies the hashing algorithm that is used for secure user credential storage. +Specifies the hashing algorithm that is used for secure user credential storage. See <>. Defaults to `bcrypt`. [float] @@ -129,7 +129,7 @@ unused for `168h` (7 days). `xpack.security.dls.bitset.cache.size`:: The maximum memory usage of cached `BitSet` entries for document level security. Document level security queries may depend on Lucene BitSet objects, and these are -automatically cached to improve performance. Defaults to `50mb`, after which +automatically cached to improve performance. Defaults to `50mb`, after which least-recently-used entries will be evicted. [float] @@ -181,8 +181,11 @@ Defaults to `ssha256`. [float] [[realm-settings]] ==== Realm settings +// tag::realm-settings-description-tag[] You configure realm settings in the `xpack.security.authc.realms` -namespace in `elasticsearch.yml`. For example: +namespace in `elasticsearch.yml`. + +For example: [source,yaml] ---------------------------------------- @@ -207,11 +210,12 @@ information is required. The valid settings vary depending on the realm type. For more information, see <>. +// end::realm-settings-description-tag[] [float] [[ref-realm-settings]] ===== Settings valid for all realms - +// tag::realm-order-tag[] `order`:: The priority of the realm within the realm chain. Realms with a lower order are consulted first. Although not required, use of this setting is strongly @@ -224,20 +228,19 @@ realm without removing its configuration information. Defaults to `true`. [[ref-native-settings]] [float] ===== Native realm settings +For a native realm, the `type` must be set to `native`. In addition to the +<>, you can specify +the following optional settings: -For a native realm, the `type` must be set to `native`. In addition to the -<>, you can specify -the following optional settings: - -`cache.ttl`:: The time-to-live for cached user entries. A user and a hash of its -credentials are cached for this period of time. Specify the time period using +`cache.ttl`:: The time-to-live for cached user entries. A user and a hash of its +credentials are cached for this period of time. Specify the time period using the standard {es} <>. Defaults to `20m`. -`cache.max_users`:: The maximum number of user entries that can live in the +`cache.max_users`:: The maximum number of user entries that can live in the cache at any given time. Defaults to 100,000. -`cache.hash_algo`:: (Expert Setting) The hashing algorithm that is used for the -in-memory cached user credentials. For possible values, see <>. +`cache.hash_algo`:: (Expert Setting) The hashing algorithm that is used for the +in-memory cached user credentials. For possible values, see <>. Defaults to `ssha256`. `authentication.enabled`:: If set to `false`, disables authentication support in @@ -251,13 +254,13 @@ Defaults to `true`. [float] ===== File realm settings -The `type` setting must be set to `file`. In addition to the -<>, you can specify -the following settings: +The `type` setting must be set to `file`. In addition to the +<>, you can specify +the following settings: `cache.ttl`:: -The time-to-live for cached user entries. A user and a hash of its credentials -are cached for this configured period of time. Defaults to `20m`. Specify values +The time-to-live for cached user entries. A user and a hash of its credentials +are cached for this configured period of time. Defaults to `20m`. Specify values using the standard {es} <>. Defaults to `20m`. @@ -279,8 +282,8 @@ Defaults to `true`. [float] ===== LDAP realm settings -The `type` setting must be set to `ldap`. In addition to the -<>, you can specify the following settings: +The `type` setting must be set to `ldap`. In addition to the +<>, you can specify the following settings: `url`:: One or more LDAP URLs in the `ldap[s]://:` format. Required. @@ -304,13 +307,13 @@ to `1h`. The DN of the user that is used to bind to the LDAP and perform searches. Only applicable in user search mode. If not specified, an anonymous bind is attempted. -Defaults to Empty. Due to its potential security impact, `bind_dn` is not +Defaults to Empty. Due to its potential security impact, `bind_dn` is not exposed via the <>. `bind_password`:: -deprecated[6.3] Use `secure_bind_password` instead. The password for the user +deprecated[6.3] Use `secure_bind_password` instead. The password for the user that is used to bind to the LDAP directory. -Defaults to Empty. Due to its potential security impact, `bind_password` is not +Defaults to Empty. Due to its potential security impact, `bind_password` is not exposed via the <>. @@ -321,7 +324,7 @@ Defaults to Empty. `user_dn_templates`:: The DN template that replaces the user name with the string `{0}`. This setting is multivalued; you can specify multiple user contexts. -Required to operate in user template mode. If `user_search.base_dn` is specified, +Required to operate in user template mode. If `user_search.base_dn` is specified, this setting is not valid. For more information on the different modes, see <>. @@ -333,19 +336,19 @@ consulted in the order that they are defined in this list. See <>. + -- -NOTE: If any settings starting with `user_search` are specified, the +NOTE: If any settings starting with `user_search` are specified, the `user_dn_templates` settings are ignored. -- `user_group_attribute`:: Specifies the attribute to examine on the user for group membership. -If any `group_search` settings are specified, this setting is ignored. Defaults +If any `group_search` settings are specified, this setting is ignored. Defaults to `memberOf`. `user_search.base_dn`:: Specifies a container DN to search for users. Required -to operated in user search mode. If `user_dn_templates` is specified, this +to operated in user search mode. If `user_dn_templates` is specified, this setting is not valid. For more information on the different modes, see <>. @@ -366,7 +369,7 @@ deprecated[5.6] Use `user_search.filter` instead. The attribute to match with the username sent with the request. Defaults to `uid`. `user_search.pool.enabled`:: -Enables or disables connection pooling for user search. If set to `false`, a new +Enables or disables connection pooling for user search. If set to `false`, a new connection is created for every search. The default is `true` when `bind_dn` is set. @@ -376,7 +379,7 @@ connection pool. Defaults to `20`. `user_search.pool.initial_size`:: The initial number of connections to create to the LDAP server on startup. -Defaults to `0`. If the LDAP server is down, values greater than `0` could cause +Defaults to `0`. If the LDAP server is down, values greater than `0` could cause startup failures. `user_search.pool.health_check.enabled`:: @@ -405,8 +408,8 @@ Specifies whether the group search should be `sub_tree`, `one_level` or `base` specifies that the `base_dn` is a group object, and that it is the only group considered. Defaults to `sub_tree`. -`group_search.filter`:: -Specifies a filter to use to look up a group. +`group_search.filter`:: +Specifies a filter to use to look up a group. When not set, the realm searches for `group`, `groupOfNames`, `groupOfUniqueNames`, or `posixGroup` with the attributes `member`, `memberOf`, or `memberUid`. Any instance of `{0}` in the filter is replaced by the user attribute defined in @@ -417,9 +420,9 @@ Specifies the user attribute that is fetched and provided as a parameter to the filter. If not set, the user DN is passed into the filter. Defaults to Empty. `unmapped_groups_as_roles`:: -If set to `true`, the names of any unmapped LDAP groups are used as role names -and assigned to the user. A group is considered to be _unmapped_ if it is not -referenced in a <>. API-based +If set to `true`, the names of any unmapped LDAP groups are used as role names +and assigned to the user. A group is considered to be _unmapped_ if it is not +referenced in a <>. API-based role mappings are not considered. Defaults to `false`. `files.role_mapping`:: @@ -522,12 +525,12 @@ include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-verification-mode-v `ssl.supported_protocols`:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-supported-protocols] -`ssl.cipher_suites`:: Specifies the cipher suites that should be supported when -communicating with the LDAP server. +`ssl.cipher_suites`:: Specifies the cipher suites that should be supported when +communicating with the LDAP server. include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-cipher-suites-values] `cache.ttl`:: -Specifies the time-to-live for cached user entries. A user and a hash of its +Specifies the time-to-live for cached user entries. A user and a hash of its credentials are cached for this period of time. Use the standard {es} <>. Defaults to `20m`. @@ -549,9 +552,9 @@ Defaults to `true`. [float] ===== Active Directory realm settings -The `type` setting must be set to `active_directory`. In addition to the -<>, you can specify -the following settings: +The `type` setting must be set to `active_directory`. In addition to the +<>, you can specify +the following settings: `url`:: One or more LDAP URLs in the `ldap[s]://:` format. Defaults to @@ -565,7 +568,7 @@ While both are supported, you can't mix the `ldap` and `ldaps` protocols. + If no URL is provided, {es} uses a default of `ldap://:389`. This default uses the `domain_name` setting value and assumes an unencrypted -connection to port 389. +connection to port 389. `load_balance.type`:: The behavior to use when there are multiple LDAP URLs defined. For supported @@ -579,18 +582,18 @@ to `1h`. `domain_name`:: The domain name of Active Directory. If the `url` and the `user_search.base_dn` -settings are not specified, the cluster can derive those values from this +settings are not specified, the cluster can derive those values from this setting. Required. `bind_dn`:: The DN of the user that is used to bind to Active Directory and perform searches. -Defaults to Empty. Due to its potential security impact, `bind_dn` is not +Defaults to Empty. Due to its potential security impact, `bind_dn` is not exposed via the <>. `bind_password`:: -deprecated[6.3] Use `secure_bind_password` instead. The password for the user -that is used to bind to Active Directory. Defaults to Empty. Due to its -potential security impact, `bind_password` is not exposed via the +deprecated[6.3] Use `secure_bind_password` instead. The password for the user +that is used to bind to Active Directory. Defaults to Empty. Due to its +potential security impact, `bind_password` is not exposed via the <>. `secure_bind_password` (<>):: @@ -598,9 +601,9 @@ The password for the user that is used to bind to Active Directory. Defaults to Empty. `unmapped_groups_as_roles`:: -If set to `true`, the names of any unmapped Active Directory groups are used as -role names and assigned to the user. A group is considered _unmapped_ when it -is not referenced in any role-mapping files. API-based role mappings are not +If set to `true`, the names of any unmapped Active Directory groups are used as +role names and assigned to the user. A group is considered _unmapped_ when it +is not referenced in any role-mapping files. API-based role mappings are not considered. Defaults to `false`. `files.role_mapping`:: @@ -621,9 +624,9 @@ only user considered. Defaults to `sub_tree`. `user_search.filter`:: Specifies a filter to use to lookup a user given a username. The default filter looks up `user` objects with either `sAMAccountName` or -`userPrincipalName`. If specified, this must be a valid LDAP user search filter. -For example `(&(objectClass=user)(sAMAccountName={0}))`. For more information, -see +`userPrincipalName`. If specified, this must be a valid LDAP user search filter. +For example `(&(objectClass=user)(sAMAccountName={0}))`. For more information, +see https://msdn.microsoft.com/en-us/library/aa746475(v=vs.85).aspx[Search Filter Syntax]. `user_search.upn_filter`:: @@ -632,7 +635,7 @@ The default filter looks up `user` objects with a matching `userPrincipalName`. If specified, this must be a valid LDAP user search filter. For example, `(&(objectClass=user)(userPrincipalName={1}))`. `{1}` is the full user principal name -provided by the user. For more information, see +provided by the user. For more information, see https://msdn.microsoft.com/en-us/library/aa746475(v=vs.85).aspx[Search Filter Syntax]. `user_search.down_level_filter`:: @@ -640,8 +643,8 @@ Specifies a filter to use to lookup a user given a down level logon name (DOMAIN\user). The default filter looks up `user` objects with a matching `sAMAccountName` in the domain provided. If specified, this must be a valid LDAP user search filter. For example, -`(&(objectClass=user)(sAMAccountName={0}))`. For more information, see -https://msdn.microsoft.com/en-us/library/aa746475(v=vs.85).aspx[Search Filter Syntax]. +`(&(objectClass=user)(sAMAccountName={0}))`. For more information, see +https://msdn.microsoft.com/en-us/library/aa746475(v=vs.85).aspx[Search Filter Syntax]. `user_search.pool.enabled`:: Enables or disables connection pooling for user search. When @@ -654,8 +657,8 @@ connection pool. Defaults to `20`. `user_search.pool.initial_size`:: The initial number of connections to create to the Active Directory server on startup. -Defaults to `0`. If the LDAP server is down, values greater than 0 -could cause startup failures. +Defaults to `0`. If the LDAP server is down, values greater than 0 +could cause startup failures. `user_search.pool.health_check.enabled`:: Enables or disables a health check on Active Directory connections in the connection @@ -664,8 +667,8 @@ Defaults to `true`. `user_search.pool.health_check.dn`:: The distinguished name to be retrieved as part of the health check. -Defaults to the value of `bind_dn` if that setting is present. Otherwise, it -defaults to the value of the `user_search.base_dn` setting. +Defaults to the value of `bind_dn` if that setting is present. Otherwise, it +defaults to the value of the `user_search.base_dn` setting. `user_search.pool.health_check.interval`:: The interval to perform background checks of connections in the pool. @@ -712,7 +715,7 @@ Defaults to `5s` (5 seconds ). `ssl.certificate`:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-certificate] + -This certificate is presented to clients when they connect. +This certificate is presented to clients when they connect. `ssl.certificate_authorities`:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-certificate-authorities] @@ -774,12 +777,12 @@ include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-verification-mode-v `ssl.supported_protocols`:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-supported-protocols] -`ssl.cipher_suites`:: Specifies the cipher suites that should be supported when +`ssl.cipher_suites`:: Specifies the cipher suites that should be supported when communicating with the Active Directory server. include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-cipher-suites-values] `cache.ttl`:: -Specifies the time-to-live for cached user entries. A user and a hash of its +Specifies the time-to-live for cached user entries. A user and a hash of its credentials are cached for this configured period of time. Use the standard Elasticsearch <>). Defaults to `20m`. @@ -799,17 +802,17 @@ this realm, so that it only supports user lookups. Defaults to `true`. `follow_referrals`:: -If set to `true`, {es} follows referrals returned by the LDAP server. -Referrals are URLs returned by the server that are to be used to continue the +If set to `true`, {es} follows referrals returned by the LDAP server. +Referrals are URLs returned by the server that are to be used to continue the LDAP operation (such as `search`). Defaults to `true`. [[ref-pki-settings]] [float] ===== PKI realm settings -The `type` setting must be set to `pki`. In addition to the -<>, you can specify -the following settings: +The `type` setting must be set to `pki`. In addition to the +<>, you can specify +the following settings: `username_pattern`:: The regular expression pattern used to extract the username from the @@ -819,7 +822,7 @@ Defaults to `CN=(.*?)(?:,\|$)`. `certificate_authorities`:: List of paths to the PEM certificate files that should be used to authenticate a user's certificate as trusted. Defaults to the trusted certificates configured -for SSL. This setting cannot be used with `truststore.path`. +for SSL. This setting cannot be used with `truststore.path`. `truststore.algorithm`:: Algorithm for the truststore. Defaults to `SunX509`. @@ -844,10 +847,10 @@ Defaults to `ES_PATH_CONF/role_mapping.yml`. The names of the realms that should be consulted for delegated authorization. If this setting is used, then the PKI realm does not perform role mapping and instead loads the user from the listed realms. -See <>. +See <>. `cache.ttl`:: -Specifies the time-to-live for cached user entries. A user and a hash of its +Specifies the time-to-live for cached user entries. A user and a hash of its credentials are cached for this period of time. Use the standard {es} <>). Defaults to `20m`. @@ -870,111 +873,156 @@ Configuring authentication delegation for PKI realms>>. [[ref-saml-settings]] [float] ===== SAML realm settings +// tag::saml-description-tag[] +The `type` setting must be set to `saml`. In addition to the +<>, you can specify +the following settings. +// end::saml-description-tag[] -The `type` setting must be set to `saml`. In addition to the -<>, you can specify -the following settings: - -`idp.entity_id`:: -The Entity ID of the SAML Identity Provider. An Entity ID is a URI with a -maximum length of 1024 characters. It can be a URL (https://idp.example.com/) or -a URN (`urn:example.com:idp`) and can be found in the configuration or the SAML +// tag::saml-idp-entity-id-tag[] +`idp.entity_id` {ess-icon}:: +The Entity ID of the SAML Identity Provider. An Entity ID is a URI with a +maximum length of 1024 characters. It can be a URL (https://idp.example.com/) or +a URN (`urn:example.com:idp`) and can be found in the configuration or the SAML metadata of the Identity Provider. +// end::saml-idp-entity-id-tag[] -`idp.metadata.path`:: +// tag::saml-idp-metadata-path-tag[] +`idp.metadata.path` {ess-icon}:: The path _(recommended)_ or URL to a SAML 2.0 metadata file describing the capabilities and configuration of the Identity Provider. If a path is provided, then it is resolved relative to the {es} config directory. If a URL is provided, then it must be either a `file` URL or a `https` URL. -{es} automatically polls this metadata resource and reloads +{es} automatically polls this metadata resource and reloads the IdP configuration when changes are detected. File based resources are polled at a frequency determined by the global {es} `resource.reload.interval.high` setting, which defaults to 5 seconds. HTTPS resources are polled at a frequency determined by the realm's `idp.metadata.http.refresh` setting. +// end::saml-idp-metadata-path-tag[] -`idp.metadata.http.refresh`:: +// tag::saml-idp-metadata-http-refresh-tag[] +`idp.metadata.http.refresh` {ess-icon}:: Controls the frequency with which `https` metadata is checked for changes. Defaults to `1h` (1 hour). +// end::saml-idp-metadata-http-refresh-tag[] -`idp.use_single_logout`:: +// tag::saml-idp-use-single-logout-tag[] +`idp.use_single_logout` {ess-icon}:: Indicates whether to utilise the Identity Provider's Single Logout service (if one exists in the IdP metadata file). Defaults to `true`. +// end::saml-idp-use-single-logout-tag[] -`sp.entity_id`:: -The Entity ID to use for this SAML Service Provider. This should be entered as a -URI. We recommend that you use the base URL of your Kibana instance. For example, -`https://kibana.example.com/`. +// tag::saml-sp-entity-id-tag[] +`sp.entity_id` {ess-icon}:: +The Entity ID to use for this SAML Service Provider. This should be entered as a +URI. We recommend that you use the base URL of your Kibana instance. For example, +`https://kibana.example.com/`. +// end::saml-sp-entity-id-tag[] -`sp.acs`:: -The URL of the Assertion Consumer Service within {kib}. Typically this is the -"api/security/v1/saml" endpoint of your Kibana server. For example, -`https://kibana.example.com/api/security/v1/saml`. +// tag::saml-sp-acs-tag[] +`sp.acs` {ess-icon}:: +The URL of the Assertion Consumer Service within {kib}. Typically this is the +"api/security/saml/callback" endpoint of your Kibana server. For example, +`https://kibana.example.com/api/security/saml/callback`. +// end::saml-sp-acs-tag[] -`sp.logout`:: -The URL of the Single Logout service within {kib}. Typically this is the -"logout" endpoint of your Kibana server. For example, +// tag::saml-sp-logout-tag[] +`sp.logout` {ess-icon}:: +The URL of the Single Logout service within {kib}. Typically this is the +"logout" endpoint of your Kibana server. For example, `https://kibana.example.com/logout`. +// end::saml-sp-logout-tag[] -`attributes.principal`:: +// tag::saml-attributes-principal-tag[] +`attributes.principal` {ess-icon}:: The Name of the SAML attribute that contains the user's principal (username). +// end::saml-attributes-principal-tag[] -`attributes.groups`:: -The Name of the SAML attribute that contains the user's groups. +// tag::saml-attributes-groups-tag[] +`attributes.groups` {ess-icon}:: +The Name of the SAML attribute that contains the user's groups. +// end::saml-attributes-groups-tag[] -`attributes.name`:: -The Name of the SAML attribute that contains the user's full name. +// tag::saml-attributes-name-tag[] +`attributes.name` {ess-icon}:: +The Name of the SAML attribute that contains the user's full name. +// end::saml-attributes-name-tag[] -`attributes.mail`:: -The Name of the SAML attribute that contains the user's email address. +// tag::saml-attributes-mail-tag[] +`attributes.mail` {ess-icon}:: +The Name of the SAML attribute that contains the user's email address. +// end::saml-attributes-mail-tag[] -`attributes.dn`:: +// tag::saml-attributes-dn-tag[] +`attributes.dn` {ess-icon}:: The Name of the SAML attribute that contains the user's X.50 -_Distinguished Name_. +_Distinguished Name_. +// end::saml-attributes-dn-tag[] -`attribute_patterns.principal`:: +// tag::saml-attributes-patterns-principal-tag[] +`attribute_patterns.principal` {ess-icon}:: A Java regular expression that is matched against the SAML attribute specified by `attributes.pattern` before it is applied to the user's _principal_ property. The attribute value must match the pattern and the value of the first -_capturing group_ is used as the principal. For example, `^([^@]+)@example\\.com$` -matches email addresses from the "example.com" domain and uses the local-part as +_capturing group_ is used as the principal. For example, `^([^@]+)@example\\.com$` +matches email addresses from the "example.com" domain and uses the local-part as the principal. +// end::saml-attributes-patterns-principal-tag[] -`attribute_patterns.groups`:: +// tag::saml-attributes-patterns-groups-tag[] +`attribute_patterns.groups` {ess-icon}:: As per `attribute_patterns.principal`, but for the _group_ property. +// end::saml-attributes-patterns-groups-tag[] -`attribute_patterns.name`:: +// tag::saml-attributes-patterns-name-tag[] +`attribute_patterns.name` {ess-icon}:: As per `attribute_patterns.principal`, but for the _name_ property. +// end::saml-attributes-patterns-name-tag[] -`attribute_patterns.mail`:: +// tag::saml-attributes-patterns-mail-tag[] +`attribute_patterns.mail` {ess-icon}:: As per `attribute_patterns.principal`, but for the _mail_ property. +// end::saml-attributes-patterns-mail-tag[] -`attribute_patterns.dn`:: +// tag::saml-attributes-patterns-dn-tag[] +`attribute_patterns.dn` {ess-icon}:: As per `attribute_patterns.principal`, but for the _dn_ property. +// end::saml-attributes-patterns-dn-tag[] -`nameid_format`:: +// tag::saml-nameid-format-tag[] +`nameid_format` {ess-icon}:: The NameID format that should be requested when asking the IdP to authenticate -the current user. Defaults to requesting _transient_ names -(`urn:oasis:names:tc:SAML:2.0:nameid-format:transient`). +the current user. Defaults to requesting _transient_ names +(`urn:oasis:names:tc:SAML:2.0:nameid-format:transient`). +// end::saml-nameid-format-tag[] -`nameid.allow_create`:: The value of the `AllowCreate` attribute of the -`NameIdPolicy` element in an authentication request. Defaults to `false`. +// tag::saml-nameid-allow-create-tag[] +`nameid.allow_create` {ess-icon}:: The value of the `AllowCreate` attribute of the +`NameIdPolicy` element in an authentication request. The default value is false. +// end::saml-nameid-allow-create-tag[] -`nameid.sp_qualifier`:: The value of the `SPNameQualifier` attribute of the -`NameIdPolicy` element in an authentication request. The default is to not +// tag::saml-nameid-sp-qualifier-tag[] +`nameid.sp_qualifier` {ess-icon}:: The value of the `SPNameQualifier` attribute of the +`NameIdPolicy` element in an authentication request. The default is to not include the `SPNameQualifier` attribute. +// end::saml-nameid-sp-qualifier-tag[] -`force_authn`:: +// tag::saml-force-authn-tag[] +`force_authn` {ess-icon}:: Specifies whether to set the `ForceAuthn` attribute when requesting that the IdP -authenticate the current user. If set to `true`, the IdP is required to verify -the user’s identity, irrespective of any existing sessions they might have. +authenticate the current user. If set to `true`, the IdP is required to verify +the user’s identity, irrespective of any existing sessions they might have. Defaults to `false`. +// end::saml-force-authn-tag[] -`populate_user_metadata`:: -Specifies whether to populate the {es} user's metadata with the values that are +// tag::saml-populate-user-metadata-tag[] +`populate_user_metadata` {ess-icon}:: +Specifies whether to populate the {es} user's metadata with the values that are provided by the SAML attributes. Defaults to `true`. +// end::saml-populate-user-metadata-tag[] `authorization_realms`:: The names of the realms that should be consulted for delegated authorization. @@ -982,12 +1030,15 @@ If this setting is used, then the SAML realm does not perform role mapping and instead loads the user from the listed realms. See <>. -`allowed_clock_skew`:: +// tag::saml-allowed-clock-skew-tag[] +`allowed_clock_skew` {ess-icon}:: The maximum amount of skew that can be tolerated between the IdP's clock and the {es} node's clock. Defaults to `3m` (3 minutes). +// end::saml-allowed-clock-skew-tag[] -`req_authn_context_class_ref`:: +// tag::saml-req-authn-context-tag[] +`req_authn_context_class_ref` {ess-icon}:: A comma separated list of Authentication Context Class Reference values to be included in the Requested Authentication Context when requesting the IdP to authenticate the current user. The Authentication Context of the corresponding @@ -995,155 +1046,182 @@ authentication response should contain at least one of the requested values. + For more information, see <>. +// end::saml-req-authn-context-tag[] [float] [[ref-saml-signing-settings]] ===== SAML realm signing settings - -If a signing key is configured (that is, either `signing.key` or -`signing.keystore.path` is set), then {es} signs outgoing SAML messages. +// tag::saml-signing-description-tag[] +If a signing key is configured (that is, either `signing.key` or +`signing.keystore.path` is set), then {es} signs outgoing SAML messages. Signing can be configured using the following settings: +// end::saml-signing-description-tag[] -`signing.saml_messages`:: +// tag::saml-signing-messages-tag[] +`signing.saml_messages` {ess-icon}:: A list of SAML message types that should be signed or `*` to sign all messages. Each element in the list should be the local name of a SAML XML Element. Supported element types are `AuthnRequest`, `LogoutRequest` and `LogoutResponse`. Only valid if `signing.key` or `signing.keystore.path` is also specified. Defaults to `*`. +// end::saml-signing-messages-tag[] -`signing.key`:: +// tag::saml-signing-key-tag[] +`signing.key` {ess-icon}:: Specifies the path to the PEM encoded private key to use for SAML message signing. `signing.key` and `signing.keystore.path` cannot be used at the same time. +// end::saml-signing-key-tag[] -`signing.secure_key_passphrase` (<>):: +`signing.secure_key_passphrase` (<>) {ess-icon}:: Specifies the passphrase to decrypt the PEM encoded private key (`signing.key`) if it is encrypted. -`signing.certificate`:: -Specifies the path to the PEM encoded certificate (or certificate chain) that -corresponds to the `signing.key`. This certificate must also be included in the -Service Provider metadata or manually configured within the IdP to allow for +// tag::saml-signing-certificate-tag[] +`signing.certificate` {ess-icon}:: +Specifies the path to the PEM encoded certificate (or certificate chain) that +corresponds to the `signing.key`. This certificate must also be included in the +Service Provider metadata or manually configured within the IdP to allow for signature validation. This setting can only be used if `signing.key` is set. +// end::saml-signing-certificate-tag[] -`signing.keystore.path`:: +// tag::saml-signing-keystore-path-tag[] +`signing.keystore.path` {ess-icon}:: The path to the keystore that contains a private key and certificate. It must be either a Java keystore (jks) or a PKCS#12 file. You cannot use this setting and `signing.key` at the same time. +// end::saml-signing-keystore-path-tag[] -`signing.keystore.type`:: +// tag::saml-signing-keystore-type-tag[] +`signing.keystore.type` {ess-icon}:: The type of the keystore in `signing.keystore.path`. -Must be either `jks` or `PKCS12`. If the keystore path ends in ".p12", ".pfx", +Must be either `jks` or `PKCS12`. If the keystore path ends in ".p12", ".pfx", or "pkcs12", this setting defaults to `PKCS12`. Otherwise, it defaults to `jks`. +// end::saml-signing-keystore-type-tag[] -`signing.keystore.alias`:: +// tag::saml-signing-keystore-alias-tag[] +`signing.keystore.alias` {ess-icon}:: Specifies the alias of the key within the keystore that should be -used for SAML message signing. If the keystore contains more than one private -key, this setting must be specified. +used for SAML message signing. If the keystore contains more than one private +key, this setting must be specified. +// end::saml-signing-keystore-alias-tag[] -`signing.keystore.secure_password` (<>):: +`signing.keystore.secure_password` (<>) {ess-icon}:: The password to the keystore in `signing.keystore.path`. -`signing.keystore.secure_key_password` (<>):: +`signing.keystore.secure_key_password` (<>) {ess-icon}:: The password for the key in the keystore (`signing.keystore.path`). Defaults to the keystore password. [float] [[ref-saml-encryption-settings]] ===== SAML realm encryption settings - -If an encryption key is configured (that is, either `encryption.key` or -`encryption.keystore.path` is set), then {es} publishes an encryption -certificate when generating metadata and attempts to decrypt incoming SAML +// tag::saml-encryption-description-tag[] +If an encryption key is configured (that is, either `encryption.key` or +`encryption.keystore.path` is set), then {es} publishes an encryption +certificate when generating metadata and attempts to decrypt incoming SAML content. Encryption can be configured using the following settings: +// end::saml-encryption-description-tag[] -`encryption.key`:: +// tag::saml-encryption-key-tag[] +`encryption.key` {ess-icon}:: Specifies the path to the PEM encoded private key to use for SAML message decryption. `encryption.key` and `encryption.keystore.path` cannot be used at the same time. +// end::saml-encryption-key-tag[] `encryption.secure_key_passphrase` (<>):: Specifies the passphrase to decrypt the PEM encoded private key (`encryption.key`) if it is encrypted. -`encryption.certificate`:: -Specifies the path to the PEM encoded certificate (or certificate chain) that is -associated with the `encryption.key`. This certificate must also be included in -the Service Provider metadata or manually configured within the IdP to enable +// tag::saml-encryption-certificate-tag[] +`encryption.certificate` {ess-icon}:: +Specifies the path to the PEM encoded certificate (or certificate chain) that is +associated with the `encryption.key`. This certificate must also be included in +the Service Provider metadata or manually configured within the IdP to enable message encryption. This setting can be used only if `encryption.key` is set. +// end::saml-encryption-certificate-tag[] -`encryption.keystore.path`:: +// tag::saml-encryption-keystore-path-tag[] +`encryption.keystore.path` {ess-icon}:: The path to the keystore that contains a private key and certificate. It must be either a Java keystore (jks) or a PKCS#12 file. You cannot use this setting and `encryption.key` at the same time. +// end::saml-encryption-keystore-path-tag[] -`encryption.keystore.type`:: +// tag::saml-encryption-keystore-type-tag[] +`encryption.keystore.type` {ess-icon}:: The type of the keystore (`encryption.keystore.path`). -Must be either `jks` or `PKCS12`. If the keystore path ends in ".p12", ".pfx", +Must be either `jks` or `PKCS12`. If the keystore path ends in ".p12", ".pfx", or "pkcs12", this setting defaults to `PKCS12`. Otherwise, it defaults to `jks`. +// end::saml-encryption-keystore-type-tag[] -`encryption.keystore.alias`:: +// tag::saml-encryption-keystore-alias-tag[] +`encryption.keystore.alias` {ess-icon}:: Specifies the alias of the key within the keystore (`encryption.keystore.path`) that should be used for SAML message decryption. If not specified, all compatible key pairs from the keystore are considered as candidate keys for decryption. +// end::saml-encryption-keystore-alias-tag[] `encryption.keystore.secure_password` (<>):: The password to the keystore (`encryption.keystore.path`). `encryption.keystore.secure_key_password` (<>):: The password for the key in the keystore (`encryption.keystore.path`). Only a -single password is supported. If you are using multiple decryption keys, +single password is supported. If you are using multiple decryption keys, they cannot have individual passwords. [float] [[ref-saml-ssl-settings]] ===== SAML realm SSL settings - -If you are loading the IdP metadata over SSL/TLS (that is, `idp.metadata.path` -is a URL using the `https` protocol), the following settings can be used to +// tag::saml-ssl-description-tag[] +If you are loading the IdP metadata over SSL/TLS (that is, `idp.metadata.path` +is a URL using the `https` protocol), the following settings can be used to configure SSL. -NOTE: These settings are not used for any purpose other than loading metadata +NOTE: These settings are not used for any purpose other than loading metadata over https. -`ssl.key`:: -include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-key-pem] -+ -If HTTP client authentication is required, it uses this file. You cannot use -this setting and `ssl.keystore.path` at the same time. +// end::saml-ssl-description-tag[] -`ssl.key_passphrase`:: +// tag::saml-ssl-key-tag[] +`ssl.key` {ess-icon}:: +include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-key-pem] +// end::saml-ssl-key-tag[] + +// tag::saml-ssl-key-passphrase-tag[] +`ssl.key_passphrase` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-key-passphrase] -+ -You cannot use this setting and `ssl.secure_key_passphrase` at the same time. +// end::saml-ssl-key-passphrase-tag[] `ssl.secure_key_passphrase` (<>):: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-secure-key-passphrase] + You cannot use this setting and `ssl.key_passphrase` at the same time. -`ssl.certificate`:: +// tag::saml-ssl-certificate-tag[] +`ssl.certificate` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-certificate] -+ -This setting can be used only if `ssl.key` is set. +// end::saml-ssl-certificate-tag[] -`ssl.certificate_authorities`:: +// tag::saml-ssl-certificate-authorities-tag[] +`ssl.certificate_authorities` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-certificate-authorities] -+ -This setting and `ssl.truststore.path` cannot be used at the same time. +// end::saml-ssl-certificate-authorities-tag[] -`ssl.keystore.path`:: +// tag::saml-ssl-keystore-path-tag[] +`ssl.keystore.path` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-path] -+ -It must be either a Java keystore (jks) or a PKCS#12 file. You cannot use this -setting and `ssl.key` at the same time. +// end::saml-ssl-keystore-path-tag[] -`ssl.keystore.type`:: +// tag::saml-ssl-keystore-type[] +`ssl.keystore.type` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-type-pkcs12] +// end::saml-ssl-keystore-type[] -`ssl.keystore.password`:: +// tag::saml-ssl-keystore-password-tag[] +`ssl.keystore.password` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-password] -+ -You cannot use this setting and `ssl.keystore.secure_password` at the same time. +// end::saml-ssl-keystore-password-tag[] `ssl.keystore.secure_password` (<>):: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-secure-password] @@ -1160,100 +1238,129 @@ time. include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-secure-key-password] You cannot use this setting and `ssl.keystore.key_password` at the same time. -`ssl.truststore.path`:: +// tag::saml-ssl-truststore-path-tag[] +`ssl.truststore.path` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-path] -+ -You cannot use this setting and `ssl.certificate_authorities` at the same time. +// end::saml-ssl-truststore-path-tag[] -`ssl.truststore.type`:: +// tag::saml-ssl-truststore-type-tag[] +`ssl.truststore.type` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-type] +// end::saml-ssl-truststore-type-tag[] -`ssl.truststore.password`:: +// tag::saml-ssl-truststore-password-tag[] +`ssl.truststore.password` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-password] -+ -You cannot use this setting and `ssl.truststore.secure_password` at the same -time. +// end::saml-ssl-truststore-password-tag[] `ssl.truststore.secure_password` (<>):: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-secure-password] + This setting cannot be used with `ssl.truststore.password`. -`ssl.verification_mode`:: -Controls the verification of certificates. +// tag::saml-ssl-verification-mode-tag[] +`ssl.verification_mode` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-verification-mode-values] +// end::saml-ssl-verification-mode-tag[] -`ssl.supported_protocols`:: +// tag::saml-ssl-supported-prototols-tag[] +`ssl.supported_protocols` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-supported-protocols] +// end::saml-ssl-supported-prototols-tag[] -`ssl.cipher_suites`:: +// tag::saml-ssl-cipher-suites-tag[] +`ssl.cipher_suites` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-cipher-suites-values] +// end::saml-ssl-cipher-suites-tag[] [float] [[ref-kerberos-settings]] ===== Kerberos realm settings - +// tag::kerberos-description-tag[] For a Kerberos realm, the `type` must be set to `kerberos`. In addition to the <>, you can specify the following settings: +// end::kerberos-description-tag[] -`keytab.path`:: Specifies the path to the Kerberos keytab file that contains the +// tag::kerberos-keytab-path-tag[] +`keytab.path` {ess-icon}:: Specifies the path to the Kerberos keytab file that contains the service principal used by this {es} node. This must be a location within the {es} configuration directory and the file must have read permissions. Required. +// end::kerberos-keytab-path-tag[] -`remove_realm_name`:: Set to `true` to remove the realm part of principal names. -Principal names in Kerberos have the form `user/instance@REALM`. If this option +// tag::kerberos-remove-realm-name-tag[] +`remove_realm_name` {ess-icon}:: Set to `true` to remove the realm part of principal names. +Principal names in Kerberos have the form `user/instance@REALM`. If this option is `true`, the realm part (`@REALM`) will not be included in the username. Defaults to `false`. +// end::kerberos-remove-realm-name-tag[] `krb.debug`:: Set to `true` to enable debug logs for the Java login module that provides support for Kerberos authentication. Defaults to `false`. -`cache.ttl`:: The time-to-live for cached user entries. A user is cached for +// tag::kerberos-cache-ttl-tag[] +`cache.ttl` {ess-icon}:: The time-to-live for cached user entries. A user is cached for this period of time. Specify the time period using the standard {es} <>. Defaults to `20m`. +// end::kerberos-cache-ttl-tag[] -`cache.max_users`:: The maximum number of user entries that can live in the +// tag::kerberos-cache-max-users-tag[] +`cache.max_users` {ess-icon}:: The maximum number of user entries that can live in the cache at any given time. Defaults to 100,000. +// end::kerberos-cache-max-users-tag[] -`authorization_realms`:: +// tag::kerberos-authorization-realms-tag[] +`authorization_realms` {ess-icon}:: The names of the realms that should be consulted for delegated authorization. If this setting is used, then the Kerberos realm does not perform role mapping and instead loads the user from the listed realms. See <>. +// end::kerberos-authorization-realms-tag[] [[ref-oidc-settings]] [float] ===== OpenID Connect realm settings - +// tag::oidc-description-tag[] In addition to the <>, you -can specify the following settings: +can specify the following settings. +// end::oidc-description-tag[] -`op.issuer`:: +// tag::oidc-op-issuer-tag[] +`op.issuer` {ess-icon}:: A verifiable Identifier for your OpenID Connect Provider. An Issuer Identifier is usually a case sensitive URL using the https scheme that contains scheme, host, and optionally, port number and path components and no query or fragment components. The value for this setting should be provided by your OpenID Connect Provider. +// end::oidc-op-issuer-tag[] -`op.authorization_endpoint`:: +// tag::oidc-op-auth-endpoint-tag[] +`op.authorization_endpoint` {ess-icon}:: The URL for the Authorization Endpoint at the OpenID Connect Provider. The value for this setting should be provided by your OpenID Connect Provider. +// end::oidc-op-auth-endpoint-tag[] -`op.token_endpoint`:: +// tag::oidc-token-endpoint-tag[] +`op.token_endpoint` {ess-icon}:: The URL for the Token Endpoint at the OpenID Connect Provider. The value for this setting should be provided by your OpenID Connect Provider. +// end::oidc-token-endpoint-tag[] -`op.userinfo_endpoint`:: +// tag::oidc-userinfo-endpoint-tag[] +`op.userinfo_endpoint` {ess-icon}:: The URL for the User Info Endpoint at the OpenID Connect Provider. The value for this setting should be provided by your OpenID Connect Provider. +// end::oidc-userinfo-endpoint-tag[] -`op.endsession_endpoint`:: +// tag::oidc-endsession-endpoint-tag[] +`op.endsession_endpoint` {ess-icon}:: The URL for the End Session Endpoint at the OpenID Connect Provider. The value for this setting should be provided by your OpenID Connect Provider. +// end::oidc-endsession-endpoint-tag[] -`op.jwkset_path`:: +// tag::oidc-op-jwkset-path-tag[] +`op.jwkset_path` {ess-icon}:: The path or URL to a JSON Web Key Set with the key material that the OpenID Connect Provider uses for signing tokens and claims responses. If a path is provided, then it is resolved relative to the {es} config @@ -1263,97 +1370,133 @@ If a URL is provided, then it must be either a `file` URL or a `https` URL. requests but will attempt to refresh the JWK upon signature verification failure, as this might indicate that the OpenID Connect Provider has rotated the signing keys. - -File based resources are polled at a frequency determined by the global {es} ++ +File-based resources are polled at a frequency determined by the global {es} `resource.reload.interval.high` setting, which defaults to 5 seconds. +// end::oidc-op-jwkset-path-tag[] -`rp.client_id`:: +// tag::rp-client-id-tag[] +`rp.client_id` {ess-icon}:: The OAuth 2.0 Client Identifier that was assigned to {es} during registration -at the OpenID Connect Provider +at the OpenID Connect Provider. +// end::rp-client-id-tag[] `rp.client_secret`(<>):: The OAuth 2.0 Client Secret that was assigned to {es} during registration -at the OpenID Connect Provider +at the OpenID Connect Provider. -`rp.redirect_uri`:: +// tag::rp-redirect-uri-tag[] +`rp.redirect_uri` {ess-icon}:: The Redirect URI within {kib}. If you want to use the authorization code flow, this is the -"api/security/oidc/callback" endpoint of your {kib} server. If you want to use the implicit flow, it is the "api/security/oidc/implicit" endpoint. +`api/security/oidc/callback` endpoint of your {kib} server. If you want to use the implicit flow, it is the `api/security/oidc/implicit` endpoint. For example, `https://kibana.example.com/api/security/oidc/callback`. +// end::rp-redirect-uri-tag[] -`rp.response_type`:: +// tag::rp-response-type-tag[] +`rp.response_type` {ess-icon}:: OAuth 2.0 Response Type value that determines the authorization processing flow to be used. Can be `code` for authorization code grant flow, or one of `id_token`, `id_token token` for the implicit flow. +// end::rp-response-type-tag[] -`rp.signature_algorithm`:: +// tag::rp-signature-algorithm-tag[] +`rp.signature_algorithm` {ess-icon}:: The signature algorithm that will be used by {es} in order to verify the signature of the id tokens it will receive from the OpenID Connect Provider. -Defaults to `RSA256` +Defaults to `RSA256`. +// end::rp-signature-algorithm-tag[] -`rp.requested_scopes`:: +// tag::rp-requested-scopes-tag[] +`rp.requested_scopes` {ess-icon}:: The scope values that will be requested by the OpenID Connect Provider as part of the Authentication Request. Optional, defaults to `openid` +// end::rp-requested-scopes-tag[] -`rp.post_logout_redirect_uri`:: +// tag::rp-post-logout-redirect-url-tag[] +`rp.post_logout_redirect_uri` {ess-icon}:: The Redirect URI (usually within {kib}) that the OpenID Connect Provider should redirect the browser to after a successful Single Logout. +// end::rp-post-logout-redirect-url-tag[] +// tag::oidc-claims-principal-tag[] `claims.principal`:: The name of the OpenID Connect claim that contains the user's principal (username). +// end::oidc-claims-principal-tag[] -`claims.groups`:: +// tag::oidc-claims-groups-tag[] +`claims.groups` {ess-icon}:: The name of the OpenID Connect claim that contains the user's groups. +// end::oidc-claims-groups-tag[] -`claims.name`:: +// tag::oidc-claims-mail-tag[] +`claims.name` {ess-icon}:: The name of the OpenID Connect claim that contains the user's full name. +// end::oidc-claims-mail-tag[] -`claims.mail`:: +// tag::oidc-claims-mail-tag[] +`claims.mail` {ess-icon}:: The name of the OpenID Connect claim that contains the user's email address. +// end::oidc-claims-mail-tag[] -`claims.dn`:: +// tag::oidc-claims-dn-tag[] +`claims.dn` {ess-icon}:: The name of the OpenID Connect claim that contains the user's X.509 _Distinguished Name_. +// end::oidc-claims-dn-tag[] -`claim_patterns.principal`:: +// tag::oidc-claim-pattern-principal-tag[] +`claim_patterns.principal` {ess-icon}:: A Java regular expression that is matched against the OpenID Connect claim specified by `claims.principal` before it is applied to the user's _principal_ property. The attribute value must match the pattern and the value of the first _capturing group_ is used as the principal. For example, `^([^@]+)@example\\.com$` matches email addresses from the "example.com" domain and uses the local-part as the principal. +// end::oidc-claim-pattern-principal-tag[] -`claim_patterns.groups`:: +// tag::oidc-claim-pattern-groups-tag[] +`claim_patterns.groups` {ess-icon}:: As per `claim_patterns.principal`, but for the _group_ property. +// end::oidc-claim-pattern-groups-tag[] -`claim_patterns.name`:: +// tag::oidc-claim-pattern-name-tag[] +`claim_patterns.name` {ess-icon}:: As per `claim_patterns.principal`, but for the _name_ property. +// end::oidc-claim-pattern-name-tag[] -`claim_patterns.mail`:: +// tag::oidc-claim-pattern-mail-tag[] +`claim_patterns.mail` {ess-icon}:: As per `claim_patterns.principal`, but for the _mail_ property. +// end::oidc-claim-pattern-mail-tag[] -`claim_patterns.dn`:: +// tag::oidc-claim-pattern-dn-tag[] +`claim_patterns.dn` {ess-icon}:: As per `claim_patterns.principal`, but for the _dn_ property. +// end::oidc-claim-pattern-dn-tag[] - -`allowed_clock_skew`:: +// tag::oidc-allowed-clock-skew-tag[] +`allowed_clock_skew` {ess-icon}:: The maximum allowed clock skew to be taken into consideration when validating id tokens with regards to their creation and expiration times. +// end::oidc-allowed-clock-skew-tag[] -`populate_user_metadata`:: +// tag::oidc-populate-user-metadata-tag[] +`populate_user_metadata` {ess-icon}:: Specifies whether to populate the {es} user's metadata with the values that are provided by the OpenID Connect claims. Defaults to `true`. +// end::oidc-populate-user-metadata-tag[] `http.proxy.host`:: Specifies the address of the proxy server that will be used by the internal http client for all back-channel communication to the OpenID Connect Provider endpoints. This includes requests to the Token Endpoint, the Userinfo Endpoint and requests to fetch the JSON Web Key Set from the OP if `op.jwkset_path` is -set as a URL. +set as a URL. `http.proxy.scheme`:: -Specifies the protocol to use to connect to the proxy server that will be -used by the http client for all back-channel communication to the OpenID -Connect Provider endpoints. Defaults to `http`. Allowed values are +Specifies the protocol to use to connect to the proxy server that will be +used by the http client for all back-channel communication to the OpenID +Connect Provider endpoints. Defaults to `http`. Allowed values are `http` or `https`. `http.proxy.port`:: @@ -1361,83 +1504,94 @@ Specifies the port of the proxy server that will be used by the http client for all backchannel communication to the OpenID Connect Provider endpoints. Defaults to `80`. -`http.connect_timeout`:: +// tag::oidc-http-connect-timeout-tag[] +`http.connect_timeout` {ess-icon}:: Controls the behavior of the http client used for back-channel communication to the OpenID Connect Provider endpoints. Specifies the timeout until a connection - is established. A value of zero means the timeout is not used. Defaults to `5s` + is established. A value of zero means the timeout is not used. Defaults to `5s`. +// end::oidc-http-connect-timeout-tag[] -`http.connection_read_timeout`:: +// tag::oidc-http-read-timeout-tag[] +`http.connection_read_timeout` {ess-icon}:: Controls the behavior of the http client used for back-channel communication to the OpenID Connect Provider endpoints. Specifies the timeout used when requesting a connection from the connection manager. Defaults to `5s` +// end::oidc-http-read-timeout-tag[] -`http.socket_timeout`:: +// tag::oidc-http-socket-timeout[] +`http.socket_timeout` {ess-icon}:: Controls the behavior of the http client used for back-channel communication to the OpenID Connect Provider endpoints. Specifies the socket timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets). Defaults to -`5s` +`5s`. +// end::oidc-http-socket-timeout[] -`http.max_connections`:: +// tag::oidc-http-max-connections-tag[] +`http.max_connections` {ess-icon}:: Controls the behavior of the http client used for back-channel communication to the OpenID Connect Provider endpoints. Specifies the maximum number of connections allowed across all endpoints. +// end::oidc-http-max-connections-tag[] -`http.max_endpoint_connections`:: +// tag::oidc-http-max-endpoint-connections-tag[] +`http.max_endpoint_connections` {ess-icon}:: Controls the behavior of the http client used for back-channel communication to the OpenID Connect Provider endpoints. Specifies the maximum number of connections allowed per endpoint. +// end::oidc-http-max-endpoint-connections-tag[] [float] [[ref-oidc-ssl-settings]] ===== OpenID Connect realm SSL settings - +// tag::oidc-ssl-description-tag[] The following settings can be used to configure SSL for all outgoing http connections to the OpenID Connect Provider endpoints. NOTE: These settings are _only_ used for the back-channel communication between {es} and the OpenID Connect Provider -`ssl.key`:: -include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-key-pem] -+ -If HTTP client authentication is required, it uses this file. You cannot use -this setting and `ssl.keystore.path` at the same time. +// end::oidc-ssl-description-tag[] -`ssl.key_passphrase`:: +// tag::oidc-ssl-key-tag[] +`ssl.key` {ess-icon}:: +include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-key-pem] +// end::oidc-ssl-key-tag[] + +// tag::oidc-ssl-key-passphrase-tag[] +`ssl.key_passphrase` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-key-passphrase] -+ -You cannot use this setting and `ssl.secure_key_passphrase` at the same -time. +// end::oidc-ssl-key-passphrase-tag[] `ssl.secure_key_passphrase` (<>):: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-secure-key-passphrase] + You cannot use this setting and `ssl.key_passphrase` at the same time. -`ssl.certificate`:: +// tag::oidc-ssl-certificate-tag[] +`ssl.certificate` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-certificate] -+ -This setting can be used only if `ssl.key` is set. +// end::oidc-ssl-certificate-tag[] -`ssl.certificate_authorities`:: +// tag::oidc-ssl-certificate-authorities-tag[] +`ssl.certificate_authorities` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-certificate-authorities] -+ -This setting and `ssl.truststore.path` cannot be used at the same time. +// end::oidc-ssl-certificate-authorities-tag[] -`ssl.keystore.path`:: +// tag::oidc-ssl-keystore-path-tag[] +`ssl.keystore.path` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-path] -+ -It must be either a Java keystore (jks) or a PKCS#12 file. You cannot use this -setting and `ssl.key` at the same time. +// end::oidc-ssl-keystore-path-tag[] -`ssl.keystore.type`:: +// tag::oidc-ssl-keystore-type-tag[] +`ssl.keystore.type` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-type-pkcs12] +// end::oidc-ssl-keystore-type-tag[] -`ssl.keystore.password`:: +// tag::oidc-ssl-keystore-password-tag[] +`ssl.keystore.password` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-password] -+ -You cannot use this setting and `ssl.keystore.secure_password` at the same time. +// end::oidc-ssl-keystore-password-tag[] `ssl.keystore.secure_password` (<>):: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-secure-password] @@ -1455,59 +1609,66 @@ include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-keystore-secure-key + You cannot use this setting and `ssl.keystore.key_password` at the same time. -`ssl.truststore.path`:: +// tag::oidc-ssl-truststore-path-tag[] +`ssl.truststore.path` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-path] -+ -You cannot use this setting and `ssl.certificate_authorities` at the same time. +// end::oidc-ssl-truststore-path-tag[] -`ssl.truststore.type`:: +// tag::oidc-ssl-truststore-type-tag[] +`ssl.truststore.type` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-type] +// end::oidc-ssl-truststore-type-tag[] -`ssl.truststore.password`:: +// tag::oidc-ssl-truststore-password-tag[] +`ssl.truststore.password` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-password] -+ -You cannot use this setting and `ssl.truststore.secure_password` at the same -time. +// end::oidc-ssl-truststore-password-tag[] `ssl.truststore.secure_password` (<>):: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-truststore-secure-password] + You cannot use this setting and `ssl.truststore.password` at the same time. -`ssl.verification_mode`:: +// tag::oidc-ssl-verification-mode-tag[] +`ssl.verification_mode` {ess-icon}:: Controls the verification of certificates. include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-verification-mode-values] +// end::oidc-ssl-verification-mode-tag[] -`ssl.supported_protocols`:: +// tag::oidc-ssl-supported-protocols-tag[] +`ssl.supported_protocols` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-supported-protocols] +// end::oidc-ssl-supported-protocols-tag[] -`ssl.cipher_suites`:: +// tag::oidc-ssl-cipher-suites-tag[] +`ssl.cipher_suites` {ess-icon}:: include::{es-repo-dir}/settings/common-defs.asciidoc[tag=ssl-cipher-suites-values] +// end::oidc-ssl-cipher-suites-tag[] [float] [[load-balancing]] ===== Load balancing and failover -The `load_balance.type` setting can have the following values: +The `load_balance.type` setting can have the following values: -* `failover`: The URLs specified are used in the order that they are specified. -The first server that can be connected to will be used for all subsequent -connections. If a connection to that server fails then the next server that a +* `failover`: The URLs specified are used in the order that they are specified. +The first server that can be connected to will be used for all subsequent +connections. If a connection to that server fails then the next server that a connection can be established to will be used for subsequent connections. * `dns_failover`: In this mode of operation, only a single URL may be specified. -This URL must contain a DNS name. The system will be queried for all IP -addresses that correspond to this DNS name. Connections to the Active Directory -or LDAP server will always be tried in the order in which they were retrieved. -This differs from `failover` in that there is no reordering of the list and if a -server has failed at the beginning of the list, it will still be tried for each +This URL must contain a DNS name. The system will be queried for all IP +addresses that correspond to this DNS name. Connections to the Active Directory +or LDAP server will always be tried in the order in which they were retrieved. +This differs from `failover` in that there is no reordering of the list and if a +server has failed at the beginning of the list, it will still be tried for each subsequent connection. -* `round_robin`: Connections will continuously iterate through the list of -provided URLs. If a server is unavailable, iterating through the list of URLs +* `round_robin`: Connections will continuously iterate through the list of +provided URLs. If a server is unavailable, iterating through the list of URLs will continue until a successful connection is made. -* `dns_round_robin`: In this mode of operation, only a single URL may be -specified. This URL must contain a DNS name. The system will be queried for all -IP addresses that correspond to this DNS name. Connections will continuously -iterate through the list of addresses. If a server is unavailable, iterating +* `dns_round_robin`: In this mode of operation, only a single URL may be +specified. This URL must contain a DNS name. The system will be queried for all +IP addresses that correspond to this DNS name. Connections will continuously +iterate through the list of addresses. If a server is unavailable, iterating through the list of URLs will continue until a successful connection is made. [float] diff --git a/docs/reference/settings/slm-settings.asciidoc b/docs/reference/settings/slm-settings.asciidoc index aab31ae5312..0d0c824a422 100644 --- a/docs/reference/settings/slm-settings.asciidoc +++ b/docs/reference/settings/slm-settings.asciidoc @@ -6,7 +6,7 @@ {slm-cap} settings ++++ -These are the settings available for configuring +These are the settings available for configuring <> ({slm-init}). ==== Cluster-level settings @@ -19,7 +19,7 @@ to the `slm-history-*` indices. Defaults to `true`. [[slm-retention-schedule]] `slm.retention_schedule`:: -(<>, <>) +(<>, <>) Controls when the <> runs. Can be a periodic or absolute time schedule. Supports all values supported by the <>. @@ -31,3 +31,8 @@ Defaults to daily at 1:30am UTC: `0 30 1 * * ?`. Limits how long {slm-init} should spend deleting old snapshots. Defaults to one hour: `1h`. +[[repositories-url-allowed]] +// tag::repositories-url-allowed[] +`repositories.url.allowed_urls` {ess-icon}:: +Specifies the <> that snapshots can be restored from. +// end::repositories-url-allowed[] diff --git a/docs/reference/setup.asciidoc b/docs/reference/setup.asciidoc index 2cdc0eeec5b..85518bf65eb 100644 --- a/docs/reference/setup.asciidoc +++ b/docs/reference/setup.asciidoc @@ -61,6 +61,8 @@ include::modules/http.asciidoc[] include::settings/ilm-settings.asciidoc[] +include::modules/indices/index_management.asciidoc[] + include::modules/indices/recovery.asciidoc[] include::modules/indices/indexing_buffer.asciidoc[] diff --git a/docs/reference/snapshot-restore/register-repository.asciidoc b/docs/reference/snapshot-restore/register-repository.asciidoc index 6dc2690abd8..69fa43a6d15 100644 --- a/docs/reference/snapshot-restore/register-repository.asciidoc +++ b/docs/reference/snapshot-restore/register-repository.asciidoc @@ -4,7 +4,8 @@ ++++ Register repository ++++ - +[[snapshots-register-repository-description]] +// tag::snapshots-register-repository-tag[] You must register a snapshot repository before you can perform snapshot and restore operations. We recommend creating a new snapshot repository for each major version. The valid repository settings depend on the repository type. @@ -12,7 +13,7 @@ major version. The valid repository settings depend on the repository type. If you register same snapshot repository with multiple clusters, only one cluster should have write access to the repository. All other clusters connected to that repository should set the repository to `readonly` mode. - +// end::snapshots-register-repository-tag[] IMPORTANT: The snapshot format can change across major versions, so if you have clusters on different versions trying to write the same repository, snapshots written by one version may not be visible to the other and the repository could