Add documentation for config file settings (#4058)
* fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 os.yml config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 os.yml config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 os.yml config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 os.yml config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 os.yml config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 os.yml config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#214 config file settings Signed-off-by: cwillum <cwmmoore@amazon.com> * Refactor settings documentation Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add more settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * remove bad commits (#5505) Signed-off-by: Stephen Crawford <steecraw@amazon.com> * Format security settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add plugin settings and dashboards settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Specify json code highlighter Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add gateway and network settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Change heading level Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Heading text change Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Fix link Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add Notifications plugin settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Implemented tech review comments for search settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Rename directory and implement latest search setting review comment Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Remove non-existent ml circuit breaker settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add file system and s3 settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Update nav order Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add security analytics settings and specify static/dynamic for security settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Reword correlation time window Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Implemented tech review comments for network and discovery settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Implemented editorial comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Clarify security settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _install-and-configure/configuring-opensearch/security-settings.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Add cross links to static and dynamic settings Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Fix link Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: cwillum <cwmmoore@amazon.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: Stephen Crawford <steecraw@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Stephen Crawford <65832608+scrawfor99@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
This commit is contained in:
Chris Moore
GitHub
parent
924918b193
commit
67cabe1ec5
|
|
@ -3,6 +3,7 @@ Anomaly Detection plugin
|
|||
Asynchronous Search plugin
|
||||
Crypto plugin
|
||||
Cross-Cluster Replication plugin
|
||||
Custom Codecs plugin
|
||||
Maps plugin
|
||||
Notebooks plugin
|
||||
Notifications plugin
|
||||
|
|
|
|||
|
|
@ -655,7 +655,7 @@ PUT /books2
|
|||
````
|
||||
{% include copy-curl.html %}
|
||||
|
||||
The preceding request is an index API rather than an analyze API. See [DYNAMIC INDEX SETTINGS]({{site.url}}{{site.baseurl}}/im-plugin/index-settings/#dynamic-index-settings) for additional details.
|
||||
The preceding request is an index API rather than an analyze API. See [Dynamic index-level index settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index-settings/#dynamic-index-level-index-settings) for additional details.
|
||||
{: .note}
|
||||
|
||||
### Response fields
|
||||
|
|
|
|||
|
|
@ -55,69 +55,10 @@ PUT _cluster/settings
|
|||
|
||||
The GET operation has no request body options. All cluster setting field parameters are optional.
|
||||
|
||||
Not all cluster settings can be updated using the cluster settings API. You will receive the error message `"setting [cluster.some.setting], not dynamically updateable"` when trying to configure these settings via the API.
|
||||
Not all cluster settings can be updated using the cluster settings API. You will receive the error message `"setting [cluster.some.setting], not dynamically updateable"` when trying to configure these settings through the API.
|
||||
{: .note }
|
||||
|
||||
The following request field parameters are compatible with the cluster API.
|
||||
|
||||
| Field | Data type | Description |
|
||||
| :--- | :--- | :--- |
|
||||
| plugins.security_analytics.enable_workflow_usage | Boolean | Supports Alerting plugin workflow integration with Security Analytics. Determines whether composite monitor workflows are generated for the Alerting plugin after creating a new threat detector in Security Analytics. By default, the setting is `true`. <br> <br> When set to `true`, composite monitor workflows based on an associated threat detector's configuration are enabled. When set to `false`, composite monitor workflows based on an associated threat detector's configuration are disabled. <br> <br> For more information about Alerting plugin workflow integration with Security Analytics, see [Integrated Alerting plugin workflows]({{site.url}}{{site.baseurl}}/security-analytics/sec-analytics-config/detectors-config/#integrated-alerting-plugin-workflows). |
|
||||
| action.auto_create_index | Boolean | Automatically creates an index if the index doesn't already exist. Also applies any index templates that are configured. Default is `true`. |
|
||||
| action.destructive_requires_name | Boolean | When set to `true`, you must specify the index name to delete an index. You cannot delete all indexes or use wildcards. Default is `true`. |
|
||||
| cluster.indices.close.enable | Boolean | Enables closing of open indexes in OpenSearch. Default is `true`. |
|
||||
| indices.recovery.max_bytes_per_sec | String | Limits the total inbound and outbound recovery traffic for each node. This applies to peer recoveries and snapshot recoveries. Default is `40mb`. If you set the recovery traffic value to less than or equal to `0mb`, rate limiting will be disabled, which causes recovery data to be transferred at the highest possible rate. |
|
||||
| indices.recovery.max_concurrent_file_chunks | Integer | The number of file chunks sent in parallel for each recovery operation. Default is `2`. |
|
||||
| indices.recovery.max_concurrent_operations | Integer | The number of operations sent in parallel for each recovery. Default is `1`. |
|
||||
| indices.recovery.max_concurrent_remote_store_streams | Integer | The number of streams to the remote repository that can be opened in parallel when recovering a remote store index. Default is `20`. |
|
||||
| logger.org.opensearch.discovery | String | Loggers accept Log4j2’s built-in log levels: `OFF`, `FATAL`, `ERROR`, `WARN`, `INFO`, `DEBUG`, and `TRACE`. Default is `INFO`. |
|
||||
| breaker.model_inference.limit | String | The limit for the trained model circuit breaker. Default is `50%` of the JVM heap. |
|
||||
| breaker.model_inference.overhead | Integer | The constant that all trained model estimations are multiplied by to determine a final estimation. Default is `1`. |
|
||||
| search.max_buckets | Integer | The maximum number of aggregation buckets allowed in a single response. Default is `65536`. |
|
||||
| search.phase_took_enabled | Boolean | Enables returning phase-level `took` time values in search responses. Default is `false`. |
|
||||
| snapshot.max_concurrent_operations | Integer | The maximum number of concurrent snapshot operations. Default is `1000`. |
|
||||
| slm.health.failed_snapshot_warn_threshold | String | The number of failed invocations since the last successful snapshot that will indicate a problem as per the health API profile. Default is five repeated failures: `5L`. |
|
||||
| indices.breaker.total.limit | String | The starting limit for the overall parent breaker. Default is `70%` of the JVM heap if `indices.breaker.total.use_real_memory` is set to `false`. Default is `95%` of the JVM heap if `indices.breaker.total.use_real_memory` is set to `true`. |
|
||||
| indices.breaker.fielddata.limit | String | The limit for the fielddata breaker. Default is `40%` of the JVM heap. |
|
||||
| indices.breaker.fielddata.overhead | Floating point | The constant that all fielddata estimations are multiplied by to determine a final estimation. Default is `1.03`. |
|
||||
| indices.breaker.request.limit | String | The limit for the request breaker. Default is `60%` of the JVM heap. |
|
||||
| indices.breaker.request.overhead | Integer | The constant that all request estimations are multiplied by to determine a final estimation. Default is `1`. |
|
||||
| network.breaker.inflight_requests.limit | String | The limit for the in-flight requests breaker. Default is `100%` of the JVM heap. |
|
||||
| network.breaker.inflight_requests.overhead | Integer/Time unit | The constant that all in-flight request estimations are multiplied by to determine a final estimation. Default is `2`. |
|
||||
| script.max_compilations_rate | String | The limit for the number of unique dynamic scripts within a defined interval that are allowed to be compiled. Default is 150 every 5 minutes: `150/5m`. |
|
||||
| cluster.default.index.refresh_interval | Time unit | Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and also support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting. |
|
||||
| cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum set, index creation fails. |
|
||||
| cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. |
|
||||
| cluster.routing.allocation.enable | String | Enables or disables allocation for specific kinds of shards: <br /> <br /> `all` – Allows shard allocation for all types of shards. <br /> <br /> `primaries` – Allows shard allocation for primary shards only. <br /> <br /> `new_primaries` – Allows shard allocation for primary shards for new indexes only. <br /> <br /> `none` – No shard allocations are allowed for any indexes. <br /> <br /> Default is `all`. |
|
||||
| cluster.routing.allocation.node_concurrent_incoming_recoveries | Integer | Configures how many concurrent incoming shard recoveries are allowed to happen on a node. Default is `2`. |
|
||||
| cluster.routing.allocation.node_concurrent_outgoing_recoveries | Integer | Configures how many concurrent outgoing shard recoveries are allowed to happen on a node. Default is `2`. |
|
||||
| cluster.routing.allocation.node_concurrent_recoveries | String | Used to set `cluster.routing.allocation.node_concurrent_incoming_recoveries` and `cluster.routing.allocation.node_concurrent_outgoing_recoveries` to the same value. |
|
||||
| cluster.routing.allocation.node_initial_primaries_recoveries | Integer | Sets the number of recoveries for unassigned primaries after a node restart. Default is `4`. |
|
||||
| cluster.routing.allocation.same_shard.host | Boolean | When set to `true`, multiple copies of a shard are prevented from being allocated to distinct nodes on the same host. Default is `false`. |
|
||||
| cluster.routing.rebalance.enable | String | Enables or disables rebalancing for specific kinds of shards: <br /> <br /> `all` – Allows shard balancing for all types of shards. <br /> <br /> `primaries` – Allows shard balancing for primary shards only. <br /> <br /> `replicas` – Allows shard balancing for replica shards only. <br /> <br /> `none` – No shard balancing is allowed for any indexes. <br /> <br /> Default is `all`. |
|
||||
| cluster.routing.allocation.allow_rebalance | String | Specifies when shard rebalancing is allowed: <br /> <br /> `always` – Always allow rebalancing. <br /> <br /> `indices_primaries_active` – Only allow rebalancing when all primaries in the cluster are allocated. <br /> <br /> `indices_all_active` – Only allow rebalancing when all shards in the cluster are allocated. <br /> <br /> Default is `indices_all_active`. |
|
||||
| cluster.routing.allocation.cluster_concurrent_rebalance | Integer | Allows you to control how many concurrent shard rebalances are allowed across a cluster. Default is `2`. |
|
||||
| cluster.routing.allocation.balance.shard | Floating point | Defines the weight factor for the total number of shards allocated per node. Default is `0.45`. |
|
||||
| cluster.routing.allocation.balance.index | Floating point | Defines the weight factor for the number of shards per index allocated on a node. Default is `0.55`. |
|
||||
| cluster.routing.allocation.balance.threshold | Floating point | The minimum optimization value of operations that should be performed. Default is `1.0`. |
|
||||
| cluster.routing.allocation.balance.prefer_primary | Boolean | When set to `true`, OpenSearch attempts to evenly distribute the primary shards between the cluster nodes. Enabling this setting does not always guarantee an equal number of primary shards on each node, especially in the event of failover. Changing this setting to `false` after it was set to `true` does not invoke redistribution of primary shards. Default is `false`.
|
||||
| cluster.routing.allocation.disk.threshold_enabled | Boolean | When set to `false`, disables the disk allocation decider. This will also remove any existing `index.blocks.read_only_allow_delete index blocks` when disabled. Default is `true`. |
|
||||
| cluster.routing.allocation.disk.watermark.low | String | Controls the low watermark for disk usage. When set to a percentage, OpenSearch will not allocate shards to nodes with that percentage of disk used. This can also be entered as ratio value, like `0.85`. Finally, this can also be set to a byte value, like `400mb`. This setting does not affect the primary shards of newly-created indexes, but will prevent their replicas from being allocated. Default is `85%`. |
|
||||
| cluster.routing.allocation.disk.watermark.high | String | Controls the high watermark. OpenSearch will attempt to relocate shards away from a node whose disk usage is above the percentage defined. This can also be entered as a ratio value, like `0.85`. Finally, this can also be set to a byte value, like `400mb`. This setting affects the allocation of all shards. Default is `90%`. |
|
||||
| cluster.routing.allocation.disk.watermark.flood_stage | String | Controls the flood stage watermark. This is a last resort to prevent nodes from running out of disk space. OpenSearch enforces a read-only index block (`index.blocks.read_only_allow_delete`) on every index that has one or more shards allocated on the node, and that has at least one disk exceeding the flood stage. The index block is released once the disk utilization falls below the high watermark. This can also be entered as a ratio value, like `0.85`. Finally, this can also be set to a byte value, like `400mb`. Default is `95%`. |
|
||||
| cluster.info.update.interval | Time unit | Sets how often OpenSearch should check disk usage for each node in the cluster. Default is `30s`. |
|
||||
| cluster.routing.allocation.include.<attribute> | Enum | Allocates shards to a node whose `attribute` has at least one of the included comma-separated values. |
|
||||
| cluster.routing.allocation.require.<attribute> | Enum | Only allocates shards to a node whose `attribute` has all of the included comma-separated values. |
|
||||
| cluster.routing.allocation.exclude.<attribute> | Enum | Does not allocate shards to a node whose `attribute` has any of the included comma-separated values. The cluster allocation settings support the following built-in attributes: <br /> <br /> `_name` – Match nodes by node name. <br /> <br /> `_host_ip` – Match nodes by host IP address. <br /> <br /> `_publish_ip` – Match nodes by publish IP address. <br /> <br /> `_ip` – Match either `_host_ip` or `_publish_ip`. <br /> <br /> `_host` – Match nodes by hostname. <br /> <br /> `_id` – Match nodes by node ID. <br /> <br /> `_tier` – Match nodes by data tier role. |
|
||||
| cluster.routing.allocation.shard_movement_strategy | Enum | Determines the order in which shards are relocated from outgoing to incoming nodes. This setting supports the following strategies: <br /> <br /> `PRIMARY_FIRST` – Primary shards are relocated first, before replica shards. This prioritization may help prevent a cluster's health status from going red if the relocating nodes fail during the process. <br /> <br /> `REPLICA_FIRST` – Replica shards are relocated first, before primary shards. This prioritization may help prevent a cluster's health status from going red when carrying out shard relocation in a mixed-version, segment-replication-enabled OpenSearch cluster. In this situation, primary shards relocated to OpenSearch nodes of a newer version could try to copy segment files to replica shards on an older version of OpenSearch, which would result in shard failure. Relocating replica shards first may help to avoid this in multi-version clusters. <br /> <br /> `NO_PREFERENCE` – The default behavior in which the order of shard relocation has no importance.
|
||||
| cluster.blocks.read_only | Boolean | Sets the entire cluster to read-only. Default is `false`. |
|
||||
| cluster.blocks.read_only_allow_delete | Boolean | Similar to `cluster.blocks.read_only` but allows you to delete indexes. |
|
||||
| cluster.max_shards_per_node | Integer | Limits the total number of primary and replica shards for the cluster. The limit is calculated as follows: `cluster.max_shards_per_node` multiplied by the number of non-frozen data nodes. Shards for closed indexes do not count toward this limit. Default is `1000`. |
|
||||
| cluster.persistent_tasks.allocation.enable | String | Enables or disables allocation for persistent tasks: <br /> <br /> `all` – Allows persistent tasks to be assigned to nodes. <br /> <br /> `none` – No allocations are allowed for persistent tasks. This does not affect persistent tasks already running. <br /> <br /> Default is `all`. |
|
||||
| cluster.persistent_tasks.allocation.recheck_interval | Time unit | The cluster manager automatically checks whether or not persistent tasks need to be assigned when the cluster state changes in a significant way. There are other factors, such as memory usage, that will affect whether or not persistent tasks are assigned to nodes but do not otherwise cause the cluster state to change. This setting defines how often assignment checks are performed in response to these factors. Default is `30 seconds`, with a minimum of `10 seconds` being required. |
|
||||
| remote_store.moving_average_window_size | Integer | The moving average window size used to calculate the rolling statistic values exposed through the [Remote Store Stats API]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/remote-store-stats-api/). Default is `20`. Minimum enforced is `5`. |
|
||||
| indices.time_series_index.default_index_merge_policy | String | This setting allows you to specify the default merge policy for time-series indexes, particularly for those with an `@timestamp` field, such as data streams. The two available options are `tiered` (default) and `log_byte_size`. We recommend using `log_byte_size` for time-series indexes to enhance the performance of range queries with the `@timestamp` field. To override the merge policy on a per-index basis, you can use the `index.merge.policy` index setting. |
|
||||
|
||||
For a listing of all cluster settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
#### Example request
|
||||
|
||||
|
|
@ -133,7 +74,7 @@ PUT _cluster/settings
|
|||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
For more information about transient settings, persistent settings, and precedence, see [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/).
|
||||
For more information about transient settings, persistent settings, and precedence, see [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/).
|
||||
|
||||
#### Example response
|
||||
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ The following table lists all available metric groups.
|
|||
|
||||
Metric | Description
|
||||
:--- |:----
|
||||
settings | A node's settings. This is a combination of the default settings, custom settings from the [configuration file]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/#configuration-file), and dynamically [updated settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/#update-cluster-settings-using-the-api).
|
||||
settings | A node's settings. This is a combination of the default settings, custom settings from the [configuration file]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/#configuration-file), and dynamically [updated settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/#updating-cluster-settings-using-the-api).
|
||||
os | Static information about the host OS, including version, processor architecture, and available/allocated processors.
|
||||
process | Contains the process ID.
|
||||
jvm | Detailed static information about the running JVM, including arguments.
|
||||
|
|
|
|||
|
|
@ -84,8 +84,8 @@ When `datastore.type` is set to `opensearch`, the following reporting settings c
|
|||
| `datastore.user` | Username | Sets the username for the metrics store |
|
||||
| `datastore.password` | String | Sets the password for the metrics store. Alternatively, this password can be configured using the `OSB_DATASTORE_PASSWORD` environment variable, which avoids storing credentials in a plain text file. The environment variable takes precedence over the config file if both define a password. |
|
||||
| `datastore.probe.cluster_version` | String | Enables automatic detection of the metrics store’s version. Default is `true`. |
|
||||
| `datastore.number_of_shards` | Integer | The number of primary shards that the `opensearch-*` indexes should have. Any updates to this setting after initial index creation will only be applied to new `opensearch-*` indexes. Default is the [OpenSearch static index value]({{site.url}}{{site.baseurl}}/im-plugin/index-settings/#static-index-settings). |
|
||||
| `datastore.number_of_replicas` | Integer | The number of replicas each primary shard in the datastore contains. Any updates to this setting after initial index creation will only be applied to new `opensearch-* `indexes. Default is the [OpenSearch static index value]({{site.url}}{{site.baseurl}}/im-plugin/index-settings/#static-index-settings). |
|
||||
| `datastore.number_of_shards` | Integer | The number of primary shards that the `opensearch-*` indexes should have. Any updates to this setting after initial index creation will only be applied to new `opensearch-*` indexes. Default is the [OpenSearch static index value]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index-settings/#static-index-level-index-settings). |
|
||||
| `datastore.number_of_replicas` | Integer | The number of replicas each primary shard in the datastore contains. Any updates to this setting after initial index creation will only be applied to new `opensearch-* `indexes. Default is the [OpenSearch static index value]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index-settings/#static-index-level-index-settings). |
|
||||
|
||||
### Examples
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ The first step in connecting your data sources to OpenSearch is to install OpenS
|
|||
|
||||
Once you have installed OpenSearch and OpenSearch Dashboards, you can use Dashboards to connect your data sources to OpenSearch and then use Dashboards to manage data sources, create index patterns based on those data sources, run queries against a specific data source, and combine visualizations in one dashboard.
|
||||
|
||||
Configuration of the [YAML files]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/#configuration-file) and installation of the `dashboards-observability` and `opensearch-sql` plugins is necessary. For more information, see [OpenSearch plugins]({{site.url}}{{site.baseurl}}/install-and-configure/plugins/).
|
||||
Configuration of the [YAML files]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/#configuration-file) and installation of the `dashboards-observability` and `opensearch-sql` plugins is necessary. For more information, see [OpenSearch plugins]({{site.url}}{{site.baseurl}}/install-and-configure/plugins/).
|
||||
|
||||
## Create a data source connection
|
||||
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ OpenSearch has built-in date formats, but you can also create your own custom fo
|
|||
|
||||
## Default format
|
||||
|
||||
As of OpenSearch 2.12, the default date format is `strict_date_time_no_millis||strict_date_optional_time||epoch_millis`. To revert the default format back to `strict_date_optional_time||epoch_millis` (the default format for OpenSearch 2.11 and earlier), set the `opensearch.experimental.optimization.datetime_formatter_caching.enabled` feature flag to `false`. For more information about enabling and disabling feature flags, see [Enabling experimental features]({{site.url}}{{site.baseurl}}/experimental/).
|
||||
As of OpenSearch 2.12, the default date format is `strict_date_time_no_millis||strict_date_optional_time||epoch_millis`. To revert the default format back to `strict_date_optional_time||epoch_millis` (the default format for OpenSearch 2.11 and earlier), set the `opensearch.experimental.optimization.datetime_formatter_caching.enabled` feature flag to `false`. For more information about enabling and disabling feature flags, see [Enabling experimental features]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/experimental/).
|
||||
|
||||
## Built-in formats
|
||||
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ nav_order: 30
|
|||
|
||||
We don't recommend changing these settings; the defaults should work well for most use cases.
|
||||
|
||||
All settings are available using the OpenSearch `_cluster/settings` operation. None require a restart, and all can be marked `persistent` or `transient`.
|
||||
All settings are available using the OpenSearch `_cluster/settings` operation. None require a restart, and all can be marked `persistent` or `transient`. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
Setting | Default | Description
|
||||
:--- | :--- | :---
|
||||
|
|
|
|||
|
|
@ -1,123 +0,0 @@
|
|||
---
|
||||
layout: default
|
||||
title: Index settings
|
||||
nav_order: 3
|
||||
has_children: true
|
||||
---
|
||||
|
||||
# Index settings
|
||||
|
||||
You can specify index settings at index creation. There are two types of index settings:
|
||||
|
||||
- [Static index settings](#static-index-settings) are settings that you cannot update while the index is open. To update a static setting, you must close the index, update the setting, and then reopen the index.
|
||||
- [Dynamic index settings](#dynamic-index-settings) are settings that you can update at any time.
|
||||
|
||||
## Specifying a setting when creating an index
|
||||
|
||||
When creating an index, you can specify its static or dynamic settings as follows:
|
||||
|
||||
```json
|
||||
PUT /testindex
|
||||
{
|
||||
"settings": {
|
||||
"index.number_of_shards": 1,
|
||||
"index.number_of_replicas": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
## Static index settings
|
||||
|
||||
The following table lists all available static index settings.
|
||||
|
||||
Setting | Description
|
||||
:--- | :---
|
||||
index.number_of_shards | The number of primary shards in the index. Default is 1.
|
||||
index.number_of_routing_shards | The number of routing shards used to split an index.
|
||||
index.shard.check_on_startup | Whether the index's shards should be checked for corruption. Available options are `false` (do not check for corruption), `checksum` (check for physical corruption), and `true` (check for both physical and logical corruption). Default is `false`.
|
||||
index.codec | Determines how the index’s stored fields are compressed and stored on disk. This setting impacts the size of the index shards and the performance of the index operations. Valid values are: <br> - `default`<br> - `best_compression`<br> - `zstd` (OpenSearch 2.9 and later)<br> - `zstd_no_dict`(OpenSearch 2.9 and later). <br>For `zstd` and `zstd_no_dict`, you can specify the compression level in the `index.codec.compression_level` setting. For more information, see [Index codec settings]({{site.url}}{{site.baseurl}}/im-plugin/index-codecs/). Optional. Default is `default`.
|
||||
index.codec.compression_level | The compression level setting provides a tradeoff between compression ratio and speed. A higher compression level results in a higher compression ratio (smaller storage size) with a tradeoff in speed (slower compression and decompression speeds lead to greater indexing and search latencies). Can only be specified if `index.codec` is set to `zstd` and `zstd_no_dict` compression levels in OpenSearch 2.9 and later. Valid values are integers in the [1, 6] range. For more information, see [Index codec settings]({{site.url}}{{site.baseurl}}/im-plugin/index-codecs/). Optional. Default is 3.
|
||||
index.routing_partition_size | The number of shards a custom routing value can go to. Routing helps an imbalanced cluster by relocating values to a subset of shards rather than a single shard. To enable routing, set this value to greater than 1 but less than `index.number_of_shards`. Default is 1.
|
||||
index.soft_deletes.retention_lease.period | The maximum amount of time to retain a shard's history of operations. Default is `12h`.
|
||||
index.load_fixed_bitset_filters_eagerly | Whether OpenSearch should preload cached filters. Available options are `true` and `false`. Default is `true`.
|
||||
index.hidden | Whether the index should be hidden. Hidden indexes are not returned as part of queries that have wildcards. Available options are `true` and `false`. Default is `false`.
|
||||
index.merge.policy | This setting controls the merge policy for the Lucene segments. The available options are `tiered` and `log_byte_size`. The default is `tiered`, but for time-series data, such as log events, we recommend that you use the `log_byte_size` merge policy, which can improve query performance when conducting range queries on the `@timestamp` field. We recommend that you not change the merge policy of an existing index. Instead, configure this setting when creating a new index.
|
||||
|
||||
## Updating a static index setting
|
||||
|
||||
You can update a static index setting only on a closed index. The following example demonstrates updating the index codec setting.
|
||||
|
||||
First, close an index:
|
||||
|
||||
```json
|
||||
POST /testindex/_close
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Then update the settings by sending a request to the `_settings` endpoint:
|
||||
|
||||
```json
|
||||
PUT /testindex/_settings
|
||||
{
|
||||
"index": {
|
||||
"codec": "zstd_no_dict",
|
||||
"codec.compression_level": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Last, reopen the index to enable read and write operations:
|
||||
|
||||
```json
|
||||
POST /testindex/_open
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
For more information about updating settings, including supported query parameters, see [Update settings]({{site.url}}{{site.baseurl}}/api-reference/index-apis/update-settings/).
|
||||
|
||||
## Dynamic index settings
|
||||
|
||||
The following table lists all available dynamic index settings.
|
||||
|
||||
Setting | Description
|
||||
:--- | :---
|
||||
index.number_of_replicas | The number of replica shards each primary shard should have. For example, if you have 4 primary shards and set `index.number_of_replicas` to 3, the index has 12 replica shards. Default is 1.
|
||||
index.auto_expand_replicas | Whether the cluster should automatically add replica shards based on the number of data nodes. Specify a lower bound and upper limit (for example, 0--9) or `all` for the upper limit. For example, if you have 5 data nodes and set `index.auto_expand_replicas` to 0--3, then the cluster does not automatically add another replica shard. However, if you set this value to `0-all` and add 2 more nodes for a total of 7, the cluster will expand to now have 6 replica shards. Default is disabled.
|
||||
index.search.idle.after | The amount of time a shard should wait for a search or get request until it goes idle. Default is `30s`.
|
||||
index.refresh_interval | How often the index should refresh, which publishes its most recent changes and makes them available for searching. Can be set to `-1` to disable refreshing. Default is `1s`.
|
||||
index.max_result_window | The maximum value of `from` + `size` for searches of the index. `from` is the starting index to search from, and `size` is the number of results to return. Default is 10000.
|
||||
index.max_inner_result_window | The maximum value of `from` + `size` that specifies the number of returned nested search hits and most relevant document aggregated during the query. `from` is the starting index to search from, and `size` is the number of top hits to return. Default is 100.
|
||||
index.max_rescore_window | The maximum value of `window_size` for rescore requests to the index. Rescore requests reorder the index's documents and return a new score, which can be more precise. Default is the same as `index.max_inner_result_window` or 10000 by default.
|
||||
index.max_docvalue_fields_search | The maximum number of `docvalue_fields` allowed in a query. Default is 100.
|
||||
index.max_script_fields | The maximum number of `script_fields` allowed in a query. Default is 32.
|
||||
index.max_ngram_diff | The maximum difference between `min_gram` and `max_gram` values for the NGramTokenizer and NGramTokenFilter. Default is 1.
|
||||
index.max_shingle_diff | The maximum difference between `max_shingle_size` and `min_shingle_size` to feed into the `shingle` token filter. Default is 3.
|
||||
index.max_refresh_listeners | The maximum number of refresh listeners each shard is allowed to have.
|
||||
index.analyze.max_token_count | The maximum number of tokens that can be returned from the `_analyze` API operation. Default is 10000.
|
||||
index.highlight.max_analyzed_offset | The number of characters a highlight request can analyze. Default is 1000000.
|
||||
index.max_terms_count | The maximum number of terms a terms query can accept. Default is 65536.
|
||||
index.max_regex_length | The maximum character length of regex that can be in a regexp query. Default is 1000.
|
||||
index.query.default_field | A field or list of fields that OpenSearch uses in queries in case a field isn't specified in the parameters.
|
||||
index.routing.allocation.enable | Specifies options for the index’s shard allocation. Available options are `all` (allow allocation for all shards), `primaries` (allow allocation only for primary shards), `new_primaries` (allow allocation only for new primary shards), and `none` (do not allow allocation). Default is `all`.
|
||||
index.routing.rebalance.enable | Enables shard rebalancing for the index. Available options are `all` (allow rebalancing for all shards), `primaries` (allow rebalancing only for primary shards), `replicas` (allow rebalancing only for replicas), and `none` (do not allow rebalancing). Default is `all`.
|
||||
index.gc_deletes | The amount of time to retain a deleted document's version number. Default is `60s`.
|
||||
index.default_pipeline | The default ingest node pipeline for the index. If the default pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline.
|
||||
index.final_pipeline | The final ingest node pipeline for the index. If the final pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline.
|
||||
|
||||
## Updating a dynamic index setting
|
||||
|
||||
You can update a dynamic index setting at any time through the API. For example, to update the refresh interval, use the following request:
|
||||
|
||||
```json
|
||||
PUT /testindex/_settings
|
||||
{
|
||||
"index": {
|
||||
"refresh_interval": "2s"
|
||||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
For more information about updating settings, including supported query parameters, see [Update settings]({{site.url}}{{site.baseurl}}/api-reference/index-apis/update-settings/).
|
||||
|
|
@ -11,7 +11,7 @@ We don't recommend changing these settings; the defaults should work well for mo
|
|||
|
||||
Index State Management (ISM) stores its configuration in the `.opendistro-ism-config` index. Don't modify this index without using the [ISM API operations]({{site.url}}{{site.baseurl}}/im-plugin/ism/api/).
|
||||
|
||||
All settings are available using the OpenSearch `_cluster/settings` operation. None require a restart, and all can be marked `persistent` or `transient`.
|
||||
All settings are available using the OpenSearch `_cluster/settings` operation. None require a restart, and all can be marked `persistent` or `transient`. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
Setting | Default | Description
|
||||
:--- | :--- | :---
|
||||
|
|
|
|||
|
|
@ -22,15 +22,15 @@ To get started with the `ip2geo` processor, the `opensearch-geospatial` plugin m
|
|||
|
||||
## Cluster settings
|
||||
|
||||
The IP2Geo data source and `ip2geo` processor node settings are listed in the following table.
|
||||
The IP2Geo data source and `ip2geo` processor node settings are listed in the following table. All settings in this table are dynamic. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
| Key | Description | Default |
|
||||
|--------------------|-------------|---------|
|
||||
| plugins.geospatial.ip2geo.datasource.endpoint | Default endpoint for creating the data source API. | Defaults to https://geoip.maps.opensearch.org/v1/geolite2-city/manifest.json. |
|
||||
| plugins.geospatial.ip2geo.datasource.update_interval_in_days | Default update interval for creating the data source API. | Defaults to 3. |
|
||||
| plugins.geospatial.ip2geo.datasource.batch_size | Maximum number of documents to ingest in a bulk request during the IP2Geo data source creation process. | Defaults to 10,000. |
|
||||
| plugins.geospatial.ip2geo.processor.cache_size | Maximum number of results that can be cached. There is only one cache used for all IP2Geo processors in each node | Defaults to 1,000. |
|
||||
|-------------------|-------------|---------|
|
||||
| `plugins.geospatial.ip2geo.datasource.endpoint` | Default endpoint for creating the data source API. | Default is `https://geoip.maps.opensearch.org/v1/geolite2-city/manifest.json`. |
|
||||
| `plugins.geospatial.ip2geo.datasource.update_interval_in_days` | Default update interval for creating the data source API. | Default is 3. |
|
||||
| `plugins.geospatial.ip2geo.datasource.batch_size` | Maximum number of documents to ingest in a bulk request during the IP2Geo data source creation process. | Default is 10,000. |
|
||||
| `plugins.geospatial.ip2geo.processor.cache_size` | Maximum number of results that can be cached. Only one cache is used for all IP2Geo processors in each node. | Default is 1,000. |
|
||||
| `plugins.geospatial.ip2geo.timeout` | The amount of time to wait for a response from the endpoint and the cluster. | Defaults to 30 seconds. |
|
||||
|
||||
## Creating the IP2Geo data source
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
---
|
||||
layout: default
|
||||
title: Configuring OpenSearch Dashboards
|
||||
nav_order: 15
|
||||
---
|
||||
|
||||
# Configuring OpenSearch Dashboards
|
||||
|
||||
OpenSearch Dashboards uses the `opensearch_dashboards.yml` configuration file to read settings when you spin up a cluster. You can find `opensearch_dashboards.yml` in `/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml` (Docker) or `/etc/opensearch-dashboards/opensearch_dashboards.yml` (most Linux distributions) on each node.
|
||||
|
||||
For information about OpenSearch Dashboards settings, see the sample [`opensearch_dashboards.yml`](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml) file.
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
---
|
||||
layout: default
|
||||
title: Availability and recovery settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 90
|
||||
---
|
||||
|
||||
# Availability and recovery settings
|
||||
|
||||
Availability and recovery settings include settings for the following:
|
||||
|
||||
- [Snapshots](#snapshot-settings)
|
||||
- [Cluster manager task throttling](#cluster-manager-task-throttling-settings)
|
||||
- [Remote-backed storage](#remote-backed-storage-settings)
|
||||
- [Search backpressure](#search-backpressure-settings)
|
||||
- [Shard indexing backpressure](#shard-indexing-backpressure-settings)
|
||||
- [Segment replication](#segment-replication-settings)
|
||||
- [Cross-cluster replication](#cross-cluster-replication-settings)
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## Snapshot settings
|
||||
|
||||
OpenSearch supports the following snapshot settings:
|
||||
|
||||
- `snapshot.max_concurrent_operations`(Dynamic, integer): The maximum number of concurrent snapshot operations. Default is `1000`.
|
||||
|
||||
### Security-related snapshot settings
|
||||
|
||||
For security-related snapshot settings, see [Security settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/security-settings/).
|
||||
|
||||
### File system settings
|
||||
|
||||
For information about Amazon S3 repository settings, see [Amazon S3]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore/#shared-file-system).
|
||||
|
||||
### Amazon S3 settings
|
||||
|
||||
For information about Amazon S3 repository settings, see [Amazon S3]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore/#amazon-s3).
|
||||
|
||||
## Cluster manager task throttling settings
|
||||
|
||||
For information about cluster manager task throttling settings, see [Setting throttling limits]({{site.url}}{{site.baseurl}}/tuning-your-cluster/cluster-manager-task-throttling/#setting-throttling-limits).
|
||||
|
||||
## Remote-backed storage settings
|
||||
|
||||
OpenSearch supports the following cluster-level remote-backed storage settings:
|
||||
|
||||
- `cluster.remote_store.translog.buffer_interval` (Dynamic, time unit): The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present.
|
||||
|
||||
- `remote_store.moving_average_window_size` (Dynamic, integer): The moving average window size used to calculate the rolling statistic values exposed through the [Remote Store Stats API]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/remote-store-stats-api/). Default is `20`. Minimum enforced is `5`.
|
||||
|
||||
For more remote-backed storage settings, see [Remote-backed storage]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/index/) and [Configuring remote-backed storage]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/index/#configuring-remote-backed-storage).
|
||||
|
||||
For remote segment backpressure settings, see [Remote segment backpressure settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/remote-segment-backpressure/#remote-segment-backpressure-settings).
|
||||
|
||||
## Search backpressure settings
|
||||
|
||||
Search backpressure is a mechanism used to identify resource-intensive search requests and cancel them when the node is under duress. For more information, see [Search backpressure settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/search-backpressure/#search-backpressure-settings).
|
||||
|
||||
## Shard indexing backpressure settings
|
||||
|
||||
Shard indexing backpressure is a smart rejection mechanism at a per-shard level that dynamically rejects indexing requests when your cluster is under strain. For more information, see shard indexing backpressure [settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/shard-indexing-settings/).
|
||||
|
||||
## Segment replication settings
|
||||
|
||||
For information about segment replication settings, see [Segment replication]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/segment-replication/index/).
|
||||
|
||||
For information about segment replication backpressure settings, see [Segment replication backpressure]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/segment-replication/backpressure/).
|
||||
|
||||
## Cross-cluster replication settings
|
||||
|
||||
For information about cross-cluster replication settings, see [Replication settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/replication-plugin/settings/).
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
---
|
||||
layout: default
|
||||
title: Circuit breaker settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 50
|
||||
---
|
||||
|
||||
# Circuit breaker settings
|
||||
|
||||
Circuit breakers prevent OpenSearch from causing a Java OutOfMemoryError. The parent circuit breaker specifies the total available amount of memory for all child circuit breakers. The child circuit breakers specify the total available amount of memory for themselves.
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## Parent circuit breaker settings
|
||||
|
||||
OpenSearch supports the following parent circuit breaker settings:
|
||||
|
||||
- `indices.breaker.total.use_real_memory` (Static, Boolean): If `true`, the parent circuit breaker considers the actual memory usage. Otherwise, the parent circuit breaker considers the amount of memory reserved by the child circuit breakers. Default is `false`.
|
||||
|
||||
- `indices.breaker.total.limit` (Dynamic, percentage): Specifies the initial memory limit for the parent circuit breaker. If `indices.breaker.total.use_real_memory` is `true`, defaults to 95% of the JVM heap. If `indices.breaker.total.use_real_memory` is `false`, defaults to 70% of the JVM heap.
|
||||
|
||||
## Field data circuit breaker settings
|
||||
|
||||
The field data circuit breaker limits the heap memory required to load a field into the field data cache. OpenSearch supports the following field data circuit breaker settings:
|
||||
|
||||
- `indices.breaker.fielddata.limit` (Dynamic, percentage): Specifies the memory limit for the field data circuit breaker. Default is 40% of the JVM heap.
|
||||
|
||||
- `indices.breaker.fielddata.overhead` (Dynamic, double): A constant by which the field data estimations are multiplied to determine the final estimation. Default is 1.03.
|
||||
|
||||
## Request circuit breaker settings
|
||||
|
||||
The request circuit breaker limits the memory required to build data structures that are needed for a request (for example, when calculating aggregations). OpenSearch supports the following request circuit breaker settings:
|
||||
|
||||
- `indices.breaker.request.limit` (Dynamic, percentage): Specifies the memory limit for the request circuit breaker. Default is 60% of the JVM heap.
|
||||
|
||||
- `indices.breaker.request.overhead` (Dynamic, double): A constant by which the request estimations are multiplied to determine the final estimation. Default is 1.
|
||||
|
||||
## In-flight request circuit breaker settings
|
||||
|
||||
The in-flight request circuit breaker limits the memory usage for all currently running incoming requests on transport and HTTP level. The memory usage for a request is based on the content length of the request and includes memory needed for the raw request and a structured object representing the request. OpenSearch supports the following in-flight request circuit breaker settings:
|
||||
|
||||
- `network.breaker.inflight_requests.limit` (Dynamic, percentage): Specifies the memory limit for the in-flight request circuit breaker. Default is 100% of JVM heap (thus, the memory usage limit for an in-flight request is determined by the memory limit of the parent circuit breaker).
|
||||
|
||||
- `network.breaker.inflight_requests.overhead` (Dynamic, double): A constant by which the in-flight request estimations are multiplied to determine the final estimation. Default is 2.
|
||||
|
||||
## Script compilation circuit breaker settings
|
||||
|
||||
The script compilation circuit breaker limits the number of inline script compilations within a time interval. OpenSearch supports the following script compilation circuit breaker setting:
|
||||
|
||||
- `script.max_compilations_rate` (Dynamic, rate): The maximum number of unique dynamic scripts compiled within a time interval for a given context. Default is 150 every 5 minutes (`150/5m`).
|
||||
|
||||
## Regular expression circuit breaker settings
|
||||
|
||||
The regular expression circuit breaker enables or disables regular expressions and limits their complexity. OpenSearch supports the following regular expression circuit breaker settings:
|
||||
|
||||
- `script.painless.regex.enabled` (Static, string): Enables regular expressions in Painless scripts.
|
||||
Valid values are:
|
||||
- `limited`: Enables regular expressions and limits their complexity using the `script.painless.regex.limit-factor` setting.
|
||||
- `true`: Enables regular expressions. Turns off the regular expression circuit breaker and does not limit regular expression complexity.
|
||||
- `false`: Disables regular expressions. If a Painless script contains a regular expression, it returns an error.
|
||||
|
||||
Default is `limited`.
|
||||
|
||||
- `script.painless.regex.limit-factor` (Static, integer): Applied only if `script.painless.regex.enabled` is set to `limited`. Limits the number of characters a regular expression in a Painless script. The character limit is calculated by multiplying the number of characters in the script input by `script.painless.regex.limit-factor`. Default is 6 (thus, if the input has 5 characters, the maximum number of characters in a regular expression is 5 · 6 = 30).
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
---
|
||||
layout: default
|
||||
title: Cluster settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 60
|
||||
---
|
||||
|
||||
# Cluster settings
|
||||
|
||||
The following settings are related to the OpenSearch cluster.
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## Cluster-level routing and allocation settings
|
||||
|
||||
OpenSearch supports the following cluster-level routing and shard allocation settings. All settings in this list are dynamic:
|
||||
|
||||
- `cluster.routing.allocation.enable` (String): Enables or disables allocation for specific kinds of shards.
|
||||
|
||||
Valid values are:
|
||||
- `all` – Allows shard allocation for all types of shards.
|
||||
- `primaries` – Allows shard allocation for primary shards only.
|
||||
- `new_primaries` – Allows shard allocation for primary shards for new indexes only.
|
||||
- `none` – No shard allocations are allowed for any indexes.
|
||||
|
||||
Default is `all`.
|
||||
|
||||
- `cluster.routing.allocation.node_concurrent_incoming_recoveries` (Integer): Configures how many concurrent incoming shard recoveries are allowed to happen on a node. Default is `2`.
|
||||
|
||||
- `cluster.routing.allocation.node_concurrent_outgoing_recoveries` (Integer): Configures how many concurrent outgoing shard recoveries are allowed to happen on a node. Default is `2`.
|
||||
|
||||
- `cluster.routing.allocation.node_concurrent_recoveries` (String): Used to set `cluster.routing.allocation.node_concurrent_incoming_recoveries` and `cluster.routing.allocation.node_concurrent_outgoing_recoveries` to the same value.
|
||||
|
||||
- `cluster.routing.allocation.node_initial_primaries_recoveries` (Integer): Sets the number of recoveries for unassigned primaries after a node restart. Default is `4`.
|
||||
|
||||
- `cluster.routing.allocation.same_shard.host` (Boolean): When set to `true`, multiple copies of a shard are prevented from being allocated to distinct nodes on the same host. Default is `false`.
|
||||
|
||||
- `cluster.routing.rebalance.enable` (String): Enables or disables rebalancing for specific kinds of shards.
|
||||
|
||||
Valid values are:
|
||||
- `all` – Allows shard balancing for all types of shards.
|
||||
- `primaries` – Allows shard balancing for primary shards only.
|
||||
- `replicas` – Allows shard balancing for replica shards only.
|
||||
- `none` – No shard balancing is allowed for any indexes.
|
||||
|
||||
Default is `all`.
|
||||
|
||||
- `cluster.routing.allocation.allow_rebalance` (String): Specifies when shard rebalancing is allowed.
|
||||
|
||||
Valid values are:
|
||||
- `always` – Always allow rebalancing.
|
||||
- `indices_primaries_active` – Only allow rebalancing when all primaries in the cluster are allocated.
|
||||
- `indices_all_active` – Only allow rebalancing when all shards in the cluster are allocated.
|
||||
|
||||
Default is `indices_all_active`.
|
||||
|
||||
- `cluster.routing.allocation.cluster_concurrent_rebalance` (Integer): Allows you to control how many concurrent shard rebalances are allowed across a cluster. Default is `2`.
|
||||
|
||||
- `cluster.routing.allocation.balance.shard` (Floating point): Defines the weight factor for the total number of shards allocated per node. Default is `0.45`.
|
||||
|
||||
- `cluster.routing.allocation.balance.index` (Floating point): Defines the weight factor for the number of shards per index allocated on a node. Default is `0.55`.
|
||||
|
||||
- `cluster.routing.allocation.balance.threshold` (Floating point): The minimum optimization value of operations that should be performed. Default is `1.0`.
|
||||
|
||||
- `cluster.routing.allocation.balance.prefer_primary` (Boolean): When set to `true`, OpenSearch attempts to evenly distribute the primary shards between the cluster nodes. Enabling this setting does not always guarantee an equal number of primary shards on each node, especially in the event of failover. Changing this setting to `false` after it was set to `true` does not invoke redistribution of primary shards. Default is `false`.
|
||||
|
||||
- `cluster.routing.allocation.disk.threshold_enabled` (Boolean): When set to `false`, disables the disk allocation decider. This will also remove any existing `index.blocks.read_only_allow_delete index blocks` when disabled. Default is `true`.
|
||||
|
||||
- `cluster.routing.allocation.disk.watermark.low` (String): Controls the low watermark for disk usage. When set to a percentage, OpenSearch will not allocate shards to nodes with that percentage of disk used. This can also be entered as ratio value, like `0.85`. Finally, this can also be set to a byte value, like `400mb`. This setting does not affect the primary shards of newly created indexes, but will prevent their replicas from being allocated. Default is `85%`.
|
||||
|
||||
- `cluster.routing.allocation.disk.watermark.high` (String): Controls the high watermark. OpenSearch will attempt to relocate shards away from a node whose disk usage is above the percentage defined. This can also be entered as a ratio value, like `0.85`. Finally, this can also be set to a byte value, like `400mb`. This setting affects the allocation of all shards. Default is `90%`.
|
||||
|
||||
- `cluster.routing.allocation.disk.watermark.flood_stage` (String): Controls the flood stage watermark. This is a last resort to prevent nodes from running out of disk space. OpenSearch enforces a read-only index block (`index.blocks.read_only_allow_delete`) on every index that has one or more shards allocated on the node and that has at least one disk exceeding the flood stage. The index block is released once the disk utilization falls below the high watermark. This can also be entered as a ratio value, like `0.85`. Finally, this can also be set to a byte value, like `400mb`. Default is `95%`.
|
||||
|
||||
- `cluster.info.update.interval` (Time unit): Sets how often OpenSearch should check disk usage for each node in the cluster. Default is `30s`.
|
||||
|
||||
- `cluster.routing.allocation.include.<attribute>` (Enum): Allocates shards to a node whose `attribute` has at least one of the included comma-separated values.
|
||||
|
||||
- `cluster.routing.allocation.require.<attribute>` (Enum): Only allocates shards to a node whose `attribute` has all of the included comma-separated values.
|
||||
|
||||
- `cluster.routing.allocation.exclude.<attribute>` (Enum): Does not allocate shards to a node whose `attribute` has any of the included comma-separated values. The cluster allocation settings support the following built-in attributes.
|
||||
|
||||
Valid values are:
|
||||
- `_name` – Match nodes by node name.
|
||||
- `_host_ip` – Match nodes by host IP address.
|
||||
- `_publish_ip` – Match nodes by publish IP address.
|
||||
- `_ip` – Match either `_host_ip` or `_publish_ip`.
|
||||
- `_host` – Match nodes by hostname.
|
||||
- `_id` – Match nodes by node ID.
|
||||
- `_tier` – Match nodes by data tier role.
|
||||
|
||||
- `cluster.routing.allocation.shard_movement_strategy` (Enum): Determines the order in which shards are relocated from outgoing to incoming nodes.
|
||||
|
||||
This setting supports the following strategies:
|
||||
- `PRIMARY_FIRST` – Primary shards are relocated first, before replica shards. This prioritization may help prevent a cluster's health status from going red if the relocating nodes fail during the process.
|
||||
- `REPLICA_FIRST` – Replica shards are relocated first, before primary shards. This prioritization may help prevent a cluster's health status from going red when carrying out shard relocation in a mixed-version, segment-replication-enabled OpenSearch cluster. In this situation, primary shards relocated to OpenSearch nodes of a newer version could try to copy segment files to replica shards on an older version of OpenSearch, which would result in shard failure. Relocating replica shards first may help to avoid this in multi-version clusters.
|
||||
- `NO_PREFERENCE` – The default behavior in which the order of shard relocation has no importance.
|
||||
|
||||
## Cluster-level shard, block, and task settings
|
||||
|
||||
OpenSearch supports the following cluster-level shard, block, and task settings:
|
||||
|
||||
- `cluster.blocks.read_only` (Boolean): Sets the entire cluster to read-only. Default is `false`.
|
||||
|
||||
- `cluster.blocks.read_only_allow_delete` (Boolean): Similar to `cluster.blocks.read_only`, but allows you to delete indexes.
|
||||
|
||||
- `cluster.max_shards_per_node` (Integer): Limits the total number of primary and replica shards for the cluster. The limit is calculated as follows: `cluster.max_shards_per_node` multiplied by the number of non-frozen data nodes. Shards for closed indexes do not count toward this limit. Default is `1000`.
|
||||
|
||||
- `cluster.persistent_tasks.allocation.enable` (String): Enables or disables allocation for persistent tasks.
|
||||
|
||||
Valid values are:
|
||||
- `all` – Allows persistent tasks to be assigned to nodes.
|
||||
- `none` – No allocations are allowed for persistent tasks. This does not affect persistent tasks already running.
|
||||
|
||||
Default is `all`.
|
||||
|
||||
- `cluster.persistent_tasks.allocation.recheck_interval` (Time unit): The cluster manager automatically checks whether persistent tasks need to be assigned when the cluster state changes in a significant way. There are other factors, such as memory usage, that will affect whether persistent tasks are assigned to nodes but do not otherwise cause the cluster state to change. This setting defines how often assignment checks are performed in response to these factors. Default is `30 seconds`, with a minimum of `10 seconds` being required.
|
||||
|
||||
## Cluster-level index settings
|
||||
|
||||
For information about index-level index settings, see [Cluster-level index settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index-settings/#cluster-level-index-settings).
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
---
|
||||
layout: default
|
||||
title: Configuration and system settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 10
|
||||
---
|
||||
|
||||
# Configuration and system settings
|
||||
|
||||
For an overview of creating an OpenSearch cluster and examples of configuration settings, see [Creating a cluster]({{site.url}}{{site.baseurl}}/tuning-your-cluster/index/). To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
OpenSearch supports the following system settings:
|
||||
|
||||
- `cluster.name` (Static, string): The cluster name.
|
||||
|
||||
- `node.name` (Static, string): A descriptive name for the node.
|
||||
|
||||
- `node.roles` (Static, list): Defines one or more roles for an OpenSearch node. Valid values are `cluster_manager`, `data`, `ingest`, `search`, `ml`, `remote_cluster_client`, and `coordinating_only`.
|
||||
|
||||
- `path.data` (Static, string): A path to the directory where your data is stored. Separate multiple locations with commas.
|
||||
|
||||
- `path.logs` (Static, string): A path to log files.
|
||||
|
||||
- `bootstrap.memory_lock` (Static, Boolean): Locks the memory at startup. We recommend setting the heap size to about half the memory available on the system and that the owner of the process is allowed to use this limit. OpenSearch doesn't perform well when the system is swapping the memory.
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
layout: default
|
||||
title: Discovery and gateway settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 30
|
||||
---
|
||||
|
||||
# Discovery and gateway settings
|
||||
|
||||
The following are settings related to discovery and local gateway.
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## Discovery settings
|
||||
|
||||
The discovery process is used when a cluster is formed. It consists of discovering nodes and electing a cluster manager node. OpenSearch supports the following discovery settings:
|
||||
|
||||
- `discovery.seed_hosts` (Static, list): The list of hosts that perform discovery when a node is started. The default list of hosts is `["127.0.0.1", "[::1]"]`.
|
||||
|
||||
- `discovery.seed_providers` (Static, list): Specifies the types of seed hosts provider to use for obtaining seed node addresses in order to start the discovery process.
|
||||
|
||||
- `discovery.type` (Static, string): By default, OpenSearch forms a multi-node cluster. Set `discovery.type` to `single-node` to form a single-node cluster.
|
||||
|
||||
- `cluster.initial_cluster_manager_nodes` (Static, list): A list of cluster-manager-eligible nodes used to bootstrap the cluster.
|
||||
|
||||
## Gateway settings
|
||||
|
||||
The local gateway stores cluster state and shard data that is used when a cluster is restarted. OpenSearch supports the following local gateway settings:
|
||||
|
||||
- `gateway.recover_after_nodes` (Static, integer): After a full cluster restart, the number of nodes that must join the cluster before recovery can begin.
|
||||
|
||||
- `gateway.recover_after_data_nodes` (Static, integer): After a full cluster restart, the number of data nodes that must join the cluster before recovery can begin.
|
||||
|
||||
- `gateway.expected_data_nodes` (Static, integer): The number of data nodes expected to exist in the cluster. After the expected number of nodes joins the cluster, recovery of local shards can begin.
|
||||
|
||||
- `gateway.recover_after_time` (Static, time value): The amount of time to wait before starting recovery if the number of data nodes is less than the expected number of data nodes.
|
||||
|
|
@ -1,11 +1,11 @@
|
|||
---
|
||||
layout: default
|
||||
title: Enabling experimental features
|
||||
nav_order: 10
|
||||
parent: OpenSearch documentation
|
||||
title: Experimental feature flags
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 120
|
||||
---
|
||||
|
||||
# Enabling experimental features
|
||||
# Experimental feature flags
|
||||
|
||||
OpenSearch releases may contain experimental features that you can enable or disable as needed. There are several methods for enabling feature flags, depending on the installation type.
|
||||
|
||||
|
|
@ -0,0 +1,192 @@
|
|||
---
|
||||
layout: default
|
||||
title: Index settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 70
|
||||
redirect_from:
|
||||
- /im-plugin/index-settings/
|
||||
---
|
||||
|
||||
# Index settings
|
||||
|
||||
Index settings can be of two types: [cluster-level settings](#cluster-level-index-settings) that affect all indexes in the cluster and [index-level settings](#index-level-index-settings) that affect individual indexes.
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## Cluster-level index settings
|
||||
|
||||
OpenSearch supports the following cluster-level index settings. All settings in this list are dynamic:
|
||||
|
||||
- `action.auto_create_index` (Boolean): Automatically creates an index if the index doesn't already exist. Also applies any index templates that are configured. Default is `true`.
|
||||
|
||||
- `action.destructive_requires_name` (Boolean): When set to `true`, you must specify the index name to delete an index. You cannot delete all indexes or use wildcards. Default is `true`.
|
||||
|
||||
- `cluster.default.index.refresh_interval` (Time unit): Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting.
|
||||
|
||||
- `cluster.minimum.index.refresh_interval` (Time unit): Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum set, index creation fails.
|
||||
|
||||
- `cluster.indices.close.enable` (Boolean): Enables closing of open indexes in OpenSearch. Default is `true`.
|
||||
|
||||
- `indices.recovery.max_bytes_per_sec` (String): Limits the total inbound and outbound recovery traffic for each node. This applies to peer recoveries and snapshot recoveries. Default is `40mb`. If you set the recovery traffic value to less than or equal to `0mb`, rate limiting will be disabled, which causes recovery data to be transferred at the highest possible rate.
|
||||
|
||||
- `indices.recovery.max_concurrent_file_chunks` (Integer): The number of file chunks sent in parallel for each recovery operation. Default is `2`.
|
||||
|
||||
- `indices.recovery.max_concurrent_operations` (Integer): The number of operations sent in parallel for each recovery. Default is `1`.
|
||||
|
||||
- `indices.recovery.max_concurrent_remote_store_streams` (Integer): The number of streams to the remote repository that can be opened in parallel when recovering a remote store index. Default is `20`.
|
||||
|
||||
- `indices.time_series_index.default_index_merge_policy` (String): This setting allows you to specify the default merge policy for time-series indexes, particularly for those with an `@timestamp` field, such as data streams. The two available options are `tiered` (default) and `log_byte_size`. Using `log_byte_size` for time-series indexes is recommended for enhancing the performance of range queries with the `@timestamp` field. To override the merge policy on a per-index basis, you can use the `index.merge.policy` index setting.
|
||||
|
||||
- `indices.fielddata.cache.size` (String): The maximum size of the field data cache. May be specified as an absolute value (for example, `8GB`) or a percentage of the node heap (for example, `50%`). This value is static so you must specify it in the `opensearch.yml` file. If you don't specify this setting, the maximum size is unlimited. This value should be smaller than the `indices.breaker.fielddata.limit`. For more information, see [Field data circuit breaker]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/circuit-breaker/#field-data-circuit-breaker-settings).
|
||||
|
||||
## Index-level index settings
|
||||
|
||||
You can specify index settings at index creation. There are two types of index settings:
|
||||
|
||||
- [Static index-level index settings](#static-index-level-index-settings) are settings that you cannot update while the index is open. To update a static setting, you must close the index, update the setting, and then reopen the index.
|
||||
- [Dynamic index-level index settings](#dynamic-index-level-index-settings) are settings that you can update at any time.
|
||||
|
||||
### Specifying a setting when creating an index
|
||||
|
||||
When creating an index, you can specify its static or dynamic settings as follows:
|
||||
|
||||
```json
|
||||
PUT /testindex
|
||||
{
|
||||
"settings": {
|
||||
"index.number_of_shards": 1,
|
||||
"index.number_of_replicas": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
### Static index-level index settings
|
||||
|
||||
OpenSearch supports the following static index-level index settings:
|
||||
|
||||
- `index.number_of_shards` (Integer): The number of primary shards in the index. Default is 1.
|
||||
|
||||
- `index.number_of_routing_shards` (Integer): The number of routing shards used to split an index.
|
||||
|
||||
- `index.shard.check_on_startup` (Boolean): Whether the index's shards should be checked for corruption. Available options are `false` (do not check for corruption), `checksum` (check for physical corruption), and `true` (check for both physical and logical corruption). Default is `false`.
|
||||
|
||||
- `index.codec` (String): Determines how the index’s stored fields are compressed and stored on disk. This setting impacts the size of the index shards and the performance of the index operations.
|
||||
|
||||
Valid values are:
|
||||
|
||||
- `default`
|
||||
- `best_compression`
|
||||
- `zstd` (OpenSearch 2.9 and later)
|
||||
- `zstd_no_dict`(OpenSearch 2.9 and later)
|
||||
|
||||
For `zstd` and `zstd_no_dict`, you can specify the compression level in the `index.codec.compression_level` setting. For more information, see [Index codec settings]({{site.url}}{{site.baseurl}}/im-plugin/index-codecs/). Optional. Default is `default`.
|
||||
|
||||
- `index.codec.compression_level` (Integer): The compression level setting provides a tradeoff between compression ratio and speed. A higher compression level results in a higher compression ratio (smaller storage size) with a tradeoff in speed (slower compression and decompression speeds lead to greater indexing and search latencies). Can only be specified if `index.codec` is set to `zstd` and `zstd_no_dict` compression levels in OpenSearch 2.9 and later. Valid values are integers in the [1, 6] range. For more information, see [Index codec settings]({{site.url}}{{site.baseurl}}/im-plugin/index-codecs/). Optional. Default is 3.
|
||||
|
||||
- `index.routing_partition_size` (Integer): The number of shards a custom routing value can go to. Routing helps an imbalanced cluster by relocating values to a subset of shards rather than a single shard. To enable routing, set this value to greater than 1 but less than `index.number_of_shards`. Default is 1.
|
||||
|
||||
- `index.soft_deletes.retention_lease.period` (Time unit): The maximum amount of time to retain a shard's history of operations. Default is `12h`.
|
||||
|
||||
- `index.load_fixed_bitset_filters_eagerly` (Boolean): Whether OpenSearch should preload cached filters. Available options are `true` and `false`. Default is `true`.
|
||||
|
||||
- `index.hidden` (Boolean): Whether the index should be hidden. Hidden indexes are not returned as part of queries that have wildcards. Available options are `true` and `false`. Default is `false`.
|
||||
|
||||
- `index.merge.policy` (String): This setting controls the merge policy for the Lucene segments. The available options are `tiered` and `log_byte_size`. The default is `tiered`, but for time-series data, such as log events, we recommend that you use the `log_byte_size` merge policy, which can improve query performance when conducting range queries on the `@timestamp` field. We recommend that you not change the merge policy of an existing index. Instead, configure this setting when creating a new index.
|
||||
|
||||
### Updating a static index setting
|
||||
|
||||
You can update a static index setting only on a closed index. The following example demonstrates updating the index codec setting.
|
||||
|
||||
First, close an index:
|
||||
|
||||
```json
|
||||
POST /testindex/_close
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Then update the settings by sending a request to the `_settings` endpoint:
|
||||
|
||||
```json
|
||||
PUT /testindex/_settings
|
||||
{
|
||||
"index": {
|
||||
"codec": "zstd_no_dict",
|
||||
"codec.compression_level": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Last, reopen the index to enable read and write operations:
|
||||
|
||||
```json
|
||||
POST /testindex/_open
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
For more information about updating settings, including supported query parameters, see [Update settings]({{site.url}}{{site.baseurl}}/api-reference/index-apis/update-settings/).
|
||||
|
||||
### Dynamic index-level index settings
|
||||
|
||||
OpenSearch supports the following dynamic index-level index settings:
|
||||
|
||||
- `index.number_of_replicas` (Integer): The number of replica shards each primary shard should have. For example, if you have 4 primary shards and set `index.number_of_replicas` to 3, the index has 12 replica shards. Default is 1.
|
||||
|
||||
- `index.auto_expand_replicas` (String): Whether the cluster should automatically add replica shards based on the number of data nodes. Specify a lower bound and upper limit (for example, 0--9) or `all` for the upper limit. For example, if you have 5 data nodes and set `index.auto_expand_replicas` to 0--3, then the cluster does not automatically add another replica shard. However, if you set this value to `0-all` and add 2 more nodes for a total of 7, the cluster will expand to now have 6 replica shards. Default is disabled.
|
||||
|
||||
- `index.search.idle.after` (Time unit): The amount of time a shard should wait for a search or get request until it goes idle. Default is `30s`.
|
||||
|
||||
- `index.refresh_interval` (Time unit): How often the index should refresh, which publishes its most recent changes and makes them available for searching. Can be set to `-1` to disable refreshing. Default is `1s`.
|
||||
|
||||
- `index.max_result_window` (Integer): The maximum value of `from` + `size` for searches of the index. `from` is the starting index to search from, and `size` is the number of results to return. Default is 10000.
|
||||
|
||||
- `index.max_inner_result_window` (Integer): The maximum value of `from` + `size` that specifies the number of returned nested search hits and most relevant document aggregated during the query. `from` is the starting index to search from, and `size` is the number of top hits to return. Default is 100.
|
||||
|
||||
- `index.max_rescore_window` (Integer): The maximum value of `window_size` for rescore requests to the index. Rescore requests reorder the index's documents and return a new score, which can be more precise. Default is the same as `index.max_inner_result_window` or 10000 by default.
|
||||
|
||||
- `index.max_docvalue_fields_search` (Integer): The maximum number of `docvalue_fields` allowed in a query. Default is 100.
|
||||
|
||||
- `index.max_script_fields` (Integer): The maximum number of `script_fields` allowed in a query. Default is 32.
|
||||
|
||||
- `index.max_ngram_diff` (Integer): The maximum difference between `min_gram` and `max_gram` values for the `NGramTokenizer` and `NGramTokenFilter`. Default is 1.
|
||||
|
||||
- `index.max_shingle_diff` (Integer): The maximum difference between `max_shingle_size` and `min_shingle_size` to feed into the `shingle` token filter. Default is 3.
|
||||
|
||||
- `index.max_refresh_listeners` (Integer): The maximum number of refresh listeners each shard is allowed to have.
|
||||
|
||||
- `index.analyze.max_token_count` (Integer): The maximum number of tokens that can be returned from the `_analyze` API operation. Default is 10000.
|
||||
|
||||
- `index.highlight.max_analyzed_offset` (Integer): The number of characters a highlight request can analyze. Default is 1000000.
|
||||
|
||||
- `index.max_terms_count` (Integer): The maximum number of terms a terms query can accept. Default is 65536.
|
||||
|
||||
- `index.max_regex_length` (Integer): The maximum character length of regex that can be in a regexp query. Default is 1000.
|
||||
|
||||
- `index.query.default_field` (List): A field or list of fields that OpenSearch uses in queries in case a field isn't specified in the parameters.
|
||||
|
||||
- `index.routing.allocation.enable` (String): Specifies options for the index’s shard allocation. Available options are `all` (allow allocation for all shards), `primaries` (allow allocation only for primary shards), `new_primaries` (allow allocation only for new primary shards), and `none` (do not allow allocation). Default is `all`.
|
||||
|
||||
- `index.routing.rebalance.enable` (String): Enables shard rebalancing for the index. Available options are `all` (allow rebalancing for all shards), `primaries` (allow rebalancing only for primary shards), `replicas` (allow rebalancing only for replicas), and `none` (do not allow rebalancing). Default is `all`.
|
||||
|
||||
- `index.gc_deletes` (Time unit): The amount of time to retain a deleted document's version number. Default is `60s`.
|
||||
|
||||
- `index.default_pipeline` (String): The default ingest node pipeline for the index. If the default pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline.
|
||||
|
||||
- `index.final_pipeline` (String): The final ingest node pipeline for the index. If the final pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline.
|
||||
|
||||
### Updating a dynamic index setting
|
||||
|
||||
You can update a dynamic index setting at any time through the API. For example, to update the refresh interval, use the following request:
|
||||
|
||||
```json
|
||||
PUT /testindex/_settings
|
||||
{
|
||||
"index": {
|
||||
"refresh_interval": "2s"
|
||||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
For more information about updating settings, including supported query parameters, see [Update settings]({{site.url}}{{site.baseurl}}/api-reference/index-apis/update-settings/).
|
||||
|
|
@ -2,38 +2,51 @@
|
|||
layout: default
|
||||
title: Configuring OpenSearch
|
||||
nav_order: 10
|
||||
has_children: true
|
||||
redirect_from:
|
||||
- /install-and-configure/configuration/
|
||||
- /install-and-configure/configuring-opensearch/
|
||||
- /opensearch/configuration/
|
||||
---
|
||||
|
||||
# Configuring OpenSearch
|
||||
|
||||
Most OpenSearch configuration can take place in the cluster settings API. Certain operations require you to modify `opensearch.yml` and restart the cluster.
|
||||
There are two types of OpenSearch settings: [dynamic](#dynamic-settings) and [static](#static-settings).
|
||||
|
||||
Whenever possible, use the cluster settings API instead; `opensearch.yml` is local to each node, whereas the API applies the setting to all nodes in the cluster. Certain settings, however, require `opensearch.yml`. In general, these settings relate to networking, cluster formation, and the local file system. To learn more, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/).
|
||||
## Dynamic settings
|
||||
|
||||
## Specify settings as environment variables
|
||||
Dynamic index settings are settings that you can update at any time. You can configure dynamic OpenSearch settings through the Cluster Settings API. For details, see [Update cluster settings using the API](#updating-cluster-settings-using-the-api).
|
||||
|
||||
Whenever possible, use the Cluster Settings API; `opensearch.yml` is local to each node, whereas the API applies the setting to all nodes in the cluster.
|
||||
{: .tip}
|
||||
|
||||
## Static settings
|
||||
|
||||
Certain operations are static and require you to modify the `opensearch.yml` [configuration file](#configuration-file) and restart the cluster. In general, these settings relate to networking, cluster formation, and the local file system. To learn more, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/).
|
||||
|
||||
## Specifying settings as environment variables
|
||||
|
||||
You can specify environment variables as arguments using `-E` when launching OpenSearch:
|
||||
|
||||
```bash
|
||||
./opensearch -Ecluster.name=opensearch-cluster -Enode.name=opensearch-node1 -Ehttp.host=0.0.0.0 -Ediscovery.type=single-node
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
## Update cluster settings using the API
|
||||
## Updating cluster settings using the API
|
||||
|
||||
The first step in changing a setting is to view the current settings:
|
||||
The first step in changing a setting is to view the current settings by sending the following request:
|
||||
|
||||
```
|
||||
```json
|
||||
GET _cluster/settings?include_defaults=true
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
For a more concise summary of non-default settings:
|
||||
For a more concise summary of non-default settings, send the following request:
|
||||
|
||||
```
|
||||
```json
|
||||
GET _cluster/settings
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Three categories of setting exist in the cluster settings API: persistent, transient, and default. Persistent settings, well, persist after a cluster restart. After a restart, OpenSearch clears transient settings.
|
||||
|
||||
|
|
@ -44,7 +57,7 @@ If you specify the same setting in multiple places, OpenSearch uses the followin
|
|||
3. Settings from `opensearch.yml`
|
||||
4. Default settings
|
||||
|
||||
To change a setting, just specify the new one as either persistent or transient. This example shows the flat settings form:
|
||||
To change a setting, use the [Cluster Settings API]({{site.url}}{{site.baseurl}}/api-reference/cluster-api/cluster-settings/) and specify the new value as either persistent or transient. This example shows the flat settings form:
|
||||
|
||||
```json
|
||||
PUT _cluster/settings
|
||||
|
|
@ -54,6 +67,7 @@ PUT _cluster/settings
|
|||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
You can also use the expanded form, which lets you copy and paste from the GET response and change existing values:
|
||||
|
||||
|
|
@ -67,9 +81,7 @@ PUT _cluster/settings
|
|||
}
|
||||
}
|
||||
```
|
||||
|
||||
For more information about the Cluster Settings API, see [Cluster settings]({{site.url}}{{site.baseurl}}/api-reference/cluster-settings/).
|
||||
|
||||
{% include copy-curl.html %}
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -89,10 +101,11 @@ action.auto_create_index: true
|
|||
compatibility.override_main_response_version: true
|
||||
```
|
||||
|
||||
The demo configuration includes a number of settings for the Security plugin that you should modify before using OpenSearch for a production workload. To learn more, see [Security]({{site.url}}{{site.baseurl}}/security/).
|
||||
The demo configuration includes a number of [settings for the Security plugin]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/security-settings/) that you should modify before using OpenSearch for a production workload. To learn more, see [Security]({{site.url}}{{site.baseurl}}/security/).
|
||||
|
||||
### (Optional) CORS header configuration
|
||||
If you are working on a client application running against an OpenSearch cluster on a different domain, you can configure headers in `opensearch.yml` to allow for developing a local application on the same machine. Use [Cross Origin Resource Sharing](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) so your application can make calls to the OpenSearch API running locally. Add the following lines in your `custom-opensearch.yml` file (note that the "-" must be the first character in each line).
|
||||
|
||||
If you are working on a client application running against an OpenSearch cluster on a different domain, you can configure headers in `opensearch.yml` to allow for developing a local application on the same machine. Use [Cross Origin Resource Sharing](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) so that your application can make calls to the OpenSearch API running locally. Add the following lines in your `custom-opensearch.yml` file (note that the "-" must be the first character in each line).
|
||||
```yml
|
||||
- http.host:0.0.0.0
|
||||
- http.port:9200
|
||||
|
|
@ -1,9 +1,11 @@
|
|||
---
|
||||
layout: default
|
||||
title: Logs
|
||||
nav_order: 60
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 120
|
||||
redirect_from:
|
||||
- /opensearch/logs/
|
||||
- /monitoring-your-cluster/logs/
|
||||
---
|
||||
|
||||
# Logs
|
||||
|
|
@ -18,7 +20,11 @@ Logs are available as `.log` (plain text) and `.json` files. Permissions for the
|
|||
|
||||
## Application logs
|
||||
|
||||
For its application logs, OpenSearch uses [Apache Log4j 2](https://logging.apache.org/log4j/2.x/) and its built-in log levels (from least to most severe) of TRACE, DEBUG, INFO, WARN, ERROR, and FATAL. The default OpenSearch log level is INFO.
|
||||
For its application logs, OpenSearch uses [Apache Log4j 2](https://logging.apache.org/log4j/2.x/) and its built-in log levels (from least to most severe). The following table describes the logging settings.
|
||||
|
||||
| Setting | Data type | Description |
|
||||
| :--- | :--- | :--- |
|
||||
| `logger.org.opensearch.discovery` | String | Loggers accept Log4j2’s built-in log levels: `OFF`, `FATAL`, `ERROR`, `WARN`, `INFO`, `DEBUG`, and `TRACE`. Default is `INFO`. |
|
||||
|
||||
Rather than changing the default log level (`logger.level`), you change the log level for individual OpenSearch modules:
|
||||
|
||||
|
|
@ -30,6 +36,7 @@ PUT /_cluster/settings
|
|||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
The easiest way to identify modules is not from the logs, which abbreviate the path (for example, `o.o.i.r`), but from the [OpenSearch source code](https://github.com/opensearch-project/opensearch/tree/master/server/src/main/java/org/opensearch).
|
||||
{: .tip }
|
||||
|
|
@ -61,6 +68,7 @@ There are other ways to change log levels:
|
|||
```yml
|
||||
logger.org.opensearch.index.reindex: debug
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
Modifying `opensearch.yml` makes the most sense if you want to reuse your logging configuration across multiple clusters or debug startup issues with a single node.
|
||||
|
||||
|
|
@ -72,6 +80,7 @@ There are other ways to change log levels:
|
|||
# Set the log level for that ID
|
||||
logger.reindex.level = debug
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
This approach is extremely flexible but requires familiarity with the [Log4j 2 property file syntax](https://logging.apache.org/log4j/2.x/manual/configuration.html#Properties). In general, the other options offer a simpler configuration experience.
|
||||
|
||||
|
|
@ -95,7 +104,6 @@ These logs rely on thresholds to define what qualifies as a "slow" search or "sl
|
|||
|
||||
```json
|
||||
GET <some-index>/_settings?include_defaults=true
|
||||
|
||||
{
|
||||
"indexing": {
|
||||
"slowlog": {
|
||||
|
|
@ -133,6 +141,7 @@ GET <some-index>/_settings?include_defaults=true
|
|||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
To enable these logs, increase one or more thresholds:
|
||||
|
||||
|
|
@ -155,6 +164,7 @@ PUT <some-index>/_settings
|
|||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
In this example, OpenSearch logs indexing operations that take 15 seconds or longer at the WARN level and operations that take between 10 and 14.*x* seconds at the INFO level. If you set a threshold to 0 seconds, OpenSearch logs all operations, which can be useful for testing whether slow logs are indeed enabled.
|
||||
|
||||
|
|
@ -176,7 +186,7 @@ OpenSearch can log CPU time and memory utilization for the top N memory-expensiv
|
|||
|
||||
Task logging is enabled dynamically through the cluster settings API:
|
||||
|
||||
```bash
|
||||
```json
|
||||
PUT _cluster/settings
|
||||
{
|
||||
"persistent" : {
|
||||
|
|
@ -184,6 +194,7 @@ PUT _cluster/settings
|
|||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Enabling task resource consumers can have an impact on search latency.
|
||||
{:.tip}
|
||||
|
|
@ -192,13 +203,14 @@ Once enabled, logs will be written to `logs/opensearch_task_detailslog.json` and
|
|||
|
||||
To configure the logging interval and the number of search tasks logged, add the following lines to `opensearch.yml`:
|
||||
|
||||
```bash
|
||||
```yaml
|
||||
# Number of expensive search tasks to log
|
||||
cluster.task.consumers.top_n.size:100
|
||||
|
||||
# Logging interval
|
||||
cluster.task.consumers.top_n.frequency:30s
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
## Deprecation logs
|
||||
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
---
|
||||
layout: default
|
||||
title: Network settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 20
|
||||
---
|
||||
|
||||
# Network settings
|
||||
|
||||
OpenSearch uses HTTP settings to configure communication with external clients through the REST API and transport settings for internal node-to-node communication within OpenSearch.
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
OpenSearch supports the following common network settings:
|
||||
|
||||
- `network.host` (Static, list): Binds an OpenSearch node to an address. Use `0.0.0.0` to include all available network interfaces, or specify an IP address assigned to a specific interface. The `network.host` setting is a combination of `network.bind_host` and `network.publish_host` if they are the same value. An alternative to `network.host` is to configure `network.bind_host` and `network.publish_host` separately as needed. See [Advanced network settings](#advanced-network-settings).
|
||||
|
||||
- `http.port` (Static, single value or range): Binds an OpenSearch node to a custom port or a range of ports for HTTP communication. You can specify an address or a range of addresses. Default is `9200-9300`.
|
||||
|
||||
- `transport.port` (Static, single value or range): Binds an OpenSearch node to a custom port for communication between nodes. You can specify an address or a range of addresses. Default is `9300-9400`.
|
||||
|
||||
## Advanced network settings
|
||||
|
||||
OpenSearch supports the following advanced network settings:
|
||||
|
||||
- `network.bind_host` (Static, list): Binds an OpenSearch node to an address or addresses for incoming connections. Default is the value in `network.host`.
|
||||
|
||||
- `network.publish_host` (Static, list): Specifies an address or addresses that an OpenSearch node publishes to other nodes in the cluster so that they can connect to it.
|
||||
|
||||
## Advanced HTTP settings
|
||||
|
||||
OpenSearch supports the following advanced network settings for HTTP communication:
|
||||
|
||||
- `http.host` (Static, list): Sets the address of an OpenSearch node for HTTP communication. The `http.host` setting is a combination of `http.bind_host` and `http.publish_host` if they are the same value. An alternative to `http.host` is to configure `http.bind_host` and `http.publish_host` separately as needed.
|
||||
|
||||
- `http.bind_host` (Static, list): Specifies an address or addresses to which an OpenSearch node binds to listen for incoming HTTP connections.
|
||||
|
||||
- `http.publish_host` (Static, list): Specifies an address or addresses that an OpenSearch node publishes to other nodes for HTTP communication.
|
||||
|
||||
## Advanced transport settings
|
||||
|
||||
OpenSearch supports the following advanced network settings for transport communication:
|
||||
|
||||
- `transport.host` (Static, list): Sets the address of an OpenSearch node for transport communication. The `transport.host` setting is a combination of `transport.bind_host` and `transport.publish_host` if they are the same value. An alternative to `transport.host` is to configure `transport.bind_host` and `transport.publish_host` separately as needed.
|
||||
|
||||
- `transport.bind_host` (Static, list): Specifies an address or addresses to which an OpenSearch node binds to listen for incoming transport connections.
|
||||
|
||||
- `transport.publish_host` (Static, list): Specifies an address or addresses that an OpenSearch node publishes to other nodes for transport communication.
|
||||
|
|
@ -0,0 +1,92 @@
|
|||
---
|
||||
layout: default
|
||||
title: Plugin settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 100
|
||||
---
|
||||
|
||||
# Plugin settings
|
||||
|
||||
The following settings are related to OpenSearch plugins.
|
||||
|
||||
## Alerting plugin settings
|
||||
|
||||
For information about alerting settings, see [Alerting settings]({{site.url}}{{site.baseurl}}/observing-your-data/alerting/settings/#alerting-settings).
|
||||
|
||||
## Anomaly Detection plugin settings
|
||||
|
||||
For information about anomaly detection settings, see [Anomaly Detection settings]({{site.url}}{{site.baseurl}}/observing-your-data/ad/settings/).
|
||||
|
||||
## Asynchronous Search plugin settings
|
||||
|
||||
For information about asynchronous search settings, see [Asynchronous Search settings]({{site.url}}{{site.baseurl}}/search-plugins/async/settings/).
|
||||
|
||||
## Cross-Cluster Replication plugin settings
|
||||
|
||||
For information about cross-cluster replication settings, see [Replication settings]({{site.url}}{{site.baseurl}}/tuning-your-cluster/replication-plugin/settings/).
|
||||
|
||||
## Geospatial plugin settings
|
||||
|
||||
For information about the Geospatial plugin's IP2Geo processor settings, see [Cluster settings]({{site.url}}{{site.baseurl}}/ingest-pipelines/processors/ip2geo/#cluster-settings).
|
||||
|
||||
## Index Management plugin settings
|
||||
|
||||
For information about index state management (ISM) settings, see [ISM settings]({{site.url}}{{site.baseurl}}/im-plugin/ism/settings/).
|
||||
|
||||
### Index rollup settings
|
||||
|
||||
For information about index rollup settings, see [Index rollup settings]({{site.url}}{{site.baseurl}}/im-plugin/index-rollups/settings/).
|
||||
|
||||
## Job Scheduler plugin settings
|
||||
|
||||
For information about the Job Scheduler plugin settings, see [Job Scheduler cluster settings]({{site.url}}{{site.baseurl}}/monitoring-your-cluster/job-scheduler/index/#job-scheduler-cluster-settings).
|
||||
|
||||
## k-NN plugin settings
|
||||
|
||||
For information about k-NN settings, see [k-NN settings]({{site.url}}{{site.baseurl}}/search-plugins/knn/settings/).
|
||||
|
||||
## ML Commons plugin settings
|
||||
|
||||
For information about machine learning settings, see [ML Commons cluster settings]({{site.url}}{{site.baseurl}}/ml-commons-plugin/cluster-settings/).
|
||||
|
||||
## Neural Search plugin settings
|
||||
|
||||
The Security Analytics plugin supports the following settings:
|
||||
|
||||
- `plugins.neural_search.hybrid_search_disabled` (Dynamic, Boolean): Disables hybrid search. Default is `false`.
|
||||
|
||||
## Notifications plugin settings
|
||||
|
||||
The Notifications plugin supports the following settings. All settings in this list are dynamic:
|
||||
|
||||
- `opensearch.notifications.core.allowed_config_types` (List): The allowed configuration types of the Notifications plugin. Use the `GET /_plugins/_notifications/features` API to retrieve the value of this setting. Configuration types include `slack`, `chime`, `microsoft_teams`, `webhook`, `email`, `sns`, `ses_account`, `smtp_account`, and `email_group`.
|
||||
|
||||
- `opensearch.notifications.core.email.minimum_header_length` (Integer): The minimum email header length. Used for email message total length validation. Default is `160`.
|
||||
|
||||
- `opensearch.notifications.core.email.size_limit` (Integer): The email size limit. Used for email message total length validation. Default is `10000000`.
|
||||
|
||||
- `opensearch.notifications.core.http.connection_timeout` (Integer): The internal HTTP client connection timeout. The client is used for webhook-based notification channels. Default is `5000`.
|
||||
|
||||
- `opensearch.notifications.core.http.host_deny_list` (List): A list of denied hosts. The HTTP client does not send notifications to webhook URLs in this list.
|
||||
|
||||
- `opensearch.notifications.core.http.max_connection_per_route` (Integer): The maximum number of HTTP connections per route of the internal HTTP client. The client is used for webhook-based notification channels. Default is `20`.
|
||||
|
||||
- `opensearch.notifications.core.http.max_connections` (Integer): The maximum number of HTTP connections of the internal HTTP client. The client is used for webhook-based notification channels. Default is `60`.
|
||||
|
||||
- `opensearch.notifications.core.http.socket_timeout` (Integer): The socket timeout configuration of the internal HTTP client. The client is used for webhook-based notification channels. Default is `50000`.
|
||||
|
||||
- `opensearch.notifications.core.tooltip_support` (Boolean): Enables tooltip support for the Notifications plugin. Use the `GET /_plugins/_notifications/features` API to retrieve the value of this setting. Default is `true`.
|
||||
|
||||
- `opensearch.notifications.general.filter_by_backend_roles` (Boolean): Enables filtering by backend roles (role-based access control for the notification channels). Default is `false`.
|
||||
|
||||
## Security plugin settings
|
||||
|
||||
For information about the Security plugin settings, see [Security settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/security-settings/).
|
||||
|
||||
## Security Analytics plugin settings
|
||||
|
||||
For information about security analytics settings, see [Security Analytics settings]({{site.url}}{{site.baseurl}}/security-analytics/settings/).
|
||||
|
||||
## SQL plugin settings
|
||||
|
||||
For information about settings related to SQL and PPL, see [SQL settings]({{site.url}}{{site.baseurl}}/search-plugins/sql/settings/).
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
---
|
||||
layout: default
|
||||
title: Search settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 80
|
||||
---
|
||||
|
||||
# Search settings
|
||||
|
||||
OpenSearch supports the following search settings:
|
||||
|
||||
- `search.max_buckets` (Dynamic, integer): The maximum number of aggregation buckets allowed in a single response. Default is `65535`.
|
||||
|
||||
- `search.phase_took_enabled` (Dynamic, Boolean): Enables returning phase-level `took` time values in search responses. Default is `false`.
|
||||
|
||||
- `search.allow_expensive_queries` (Dynamic, Boolean): Allows or disallows expensive queries. For more information, see [Expensive queries]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries).
|
||||
|
||||
- `search.default_allow_partial_results` (Dynamic, Boolean): A cluster-level setting that allows returning partial search results if a request times out or a shard fails. If a search request contains an `allow_partial_search_results` parameter, the parameter takes precedence over this setting. Default is `true`.
|
||||
|
||||
- `search.cancel_after_time_interval` (Dynamic, time unit): A cluster-level setting that specifies the maximum amount of time that a search request can run before it is canceled at the shard level. After this time has been reached, a request is stopped and all associated tasks are canceled. Default is `-1`.
|
||||
|
||||
- `search.default_search_timeout` (Dynamic, time unit): A cluster-level setting that sets the default timeout for all search requests at the coordinating node level. If the `timeout` is specified in the search request, it takes precedence over this setting. Default is `-1` (no timeout).
|
||||
|
||||
- `search.default_keep_alive` (Dynamic, time unit): Specifies the default keep alive value for scroll and Point in Time (PIT) searches. Because a request may land on a shard multiple times (for example, during the query and fetch phases), OpenSearch opens a _request context_ that exists for the full duration of the request to ensure consistency of the shard state for each individual shard request. In a standard search, once the fetch phase completes, the request context is closed. For a scroll or a PIT search, OpenSearch keeps the request context open until explicitly closed (or until the keep alive time is reached). A background thread periodically checks all open scroll and PIT contexts and deletes the ones that have exceeded their keep alive timeout. The `search.keep_alive_interval` setting specifies how frequently the contexts are checked for expiration. The `search.default_keep_alive` setting is the default deadline for expiration. A scroll or PIT request can explicitly specify the keep alive, which takes precedence over this setting. Default is `5m`.
|
||||
|
||||
- `search.keep_alive_interval` (Static, time unit): Determines the interval at which OpenSearch checks for request contexts that have exceeded their keep alive limit. Default is `1m`.
|
||||
|
||||
- `search.max_keep_alive` (Dynamic, time unit): Specifies the maximum keep alive value. The `max_keep_alive` setting is used as a safety check against the other `keep_alive` settings (for example, `default_keep_alive`) and request-level keep alive settings (for scroll and PIT contexts). If a request exceeds the `max_keep_alive` value in either case, the operation will fail. Default is `24h`.
|
||||
|
||||
- `search.low_level_cancellation` (Dynamic, Boolean): Enables low-level request cancellation. Lucene's classic timeout mechanism only checks the time while collecting search results. However, an expensive query, such as wildcard or prefix, can take a long time to expand before starting to collect results. In this case, the query can run for a period of time that is greater than the timeout value. The low-level cancellation mechanism addresses this scenario by timing out not only while collecting search results but also during the query expansion phase or before performing any Lucene operation. Default is `true`.
|
||||
|
||||
- `search.max_open_scroll_context` (Dynamic, integer): A node-level setting that specifies the maximum number of open scroll contexts for the node. Default is `500`.
|
||||
|
||||
- `search.request_stats_enabled` (Dynamic, Boolean): Turns on node-level collection of phase-timing statistics from the perspective of the coordinator node. The request-level statistics keep track of how long (in total) search requests spend in each of the different search phases. You can retrieve these counters using the [Nodes Stats API]({{site.url}}{{site.baseurl}}/api-reference/nodes-apis/nodes-stats/). Default is `false`.
|
||||
|
||||
- `search.highlight.term_vector_multi_value` (Static, Boolean): Specifies to highlight snippets across values of a multi-valued field. Default is `true`.
|
||||
|
||||
## Point in Time settings
|
||||
|
||||
For information about PIT settings, see [PIT settings]({{site.url}}{{site.baseurl}}/search-plugins/point-in-time-api/#pit-settings).
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
|
@ -0,0 +1,483 @@
|
|||
---
|
||||
layout: default
|
||||
title: Security settings
|
||||
parent: Configuring OpenSearch
|
||||
nav_order: 40
|
||||
---
|
||||
|
||||
# Security settings
|
||||
|
||||
The Security plugin provides a number of YAML configuration files that are used to store the necessary settings that define the way the Security plugin manages users, roles, and activity within the cluster. For a full list of the Security plugin configuration files, see [Modifying the YAML files]({{site.url}}{{site.baseurl}}/security/configuration/yaml/).
|
||||
|
||||
The following sections describe security-related settings in `opensearch.yml`. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## Common settings
|
||||
|
||||
The Security plugin supports the following common settings:
|
||||
|
||||
- `plugins.security.nodes_dn` (Static): Specifies a list of distinguished names (DNs) that denote the other nodes in the cluster. This setting supports wildcards and regular expressions. The list of DNs are also read from the security index **in addition** to the YAML configuration when `plugins.security.nodes_dn_dynamic_config_enabled` is `true`.
|
||||
|
||||
- `plugins.security.nodes_dn_dynamic_config_enabled` (Static): Relevant for `cross_cluster` use cases where there is a need to manage the allow listed `nodes_dn` without having to restart the nodes every time a new `cross_cluster` remote is configured.
|
||||
Setting `nodes_dn_dynamic_config_enabled` to `true` enables **super-admin callable** Distinguished Names APIs, which provide means to update or retrieve `nodes_dn` dynamically. This setting only has effect if `plugins.security.cert.intercluster_request_evaluator_class` is not set. Default is `false`.
|
||||
|
||||
- `plugins.security.authcz.admin_dn` (Static): Defines the DNs of certificates to which admin privileges should be assigned. Required.
|
||||
|
||||
- `plugins.security.roles_mapping_resolution` (Static): Defines how backend roles are mapped to Security roles.
|
||||
|
||||
Valid values are:
|
||||
- `MAPPING_ONLY`(Default): Mappings must be configured explicitly in `roles_mapping.yml`.
|
||||
- `BACKENDROLES_ONLY`: Backend roles are mapped to security roles directly. Settings in `roles_mapping.yml` have no effect.
|
||||
- `BOTH`: Backend roles are mapped to security roles both directly and through `roles_mapping.yml`.
|
||||
|
||||
- `plugins.security.dls.mode` (Static): Sets the document-level security (DLS) evaluation mode. Default is `adaptive`. See [How to set the DLS evaluation mode]({{site.url}}{{site.baseurl}}/security/access-control/document-level-security/#how-to-set-the-dls-evaluation-mode-in-opensearchyml).
|
||||
|
||||
- `plugins.security.compliance.salt` (Static): The salt to use when generating the hash value for field masking. Must be at least 32 characters. Only ASCII characters are allowed. Optional.
|
||||
|
||||
- `config.dynamic.http.anonymous_auth_enabled` (Static): Enables anonymous authentication. This will cause all HTTP authenticators to not challenge. Default is `false`.
|
||||
|
||||
## REST management API settings
|
||||
|
||||
The Security plugin supports the following REST management API settings:
|
||||
|
||||
- `plugins.security.restapi.roles_enabled` (Static): Enables role-based access to the REST management API for listed roles. Roles are separated by a comma. Default is an empty list (no role is allowed to access the REST management API). See [Access control for the API]({{site.url}}{{site.baseurl}}/security/access-control/api/#access-control-for-the-api).
|
||||
|
||||
- `plugins.security.restapi.endpoints_disabled.<role>.<endpoint>` (Static): Disables specific endpoints and their HTTP methods for roles. Values for this setting compose an array of HTTP methods. For example: `plugins.security.restapi.endpoints_disabled.all_access.ACTIONGROUPS: ["PUT","POST","DELETE"]`. By default, all endpoints and methods are allowed. Existing endpoints include `ACTIONGROUPS`, `CACHE`, `CONFIG`, `ROLES`, `ROLESMAPPING`, `INTERNALUSERS`, `SYSTEMINFO`, `PERMISSIONSINFO`, and `LICENSE`. See [Access control for the API]({{site.url}}{{site.baseurl}}/security/access-control/api/#access-control-for-the-api).
|
||||
|
||||
- `plugins.security.restapi.password_validation_regex` (Static): Specifies a regular expression to set the criteria for the login password. For more information, see [Password settings]({{site.url}}{{site.baseurl}}/security/configuration/yaml/#password-settings).
|
||||
|
||||
- `plugins.security.restapi.password_validation_error_message` (Static): Specifies an error message that loads when a password doesn’t pass validation. This setting is used in conjunction with `plugins.security.restapi.password_validation_regex`.
|
||||
|
||||
- `plugins.security.restapi.password_min_length` (Static): Sets the minimum number of characters for the password length when using the score-based password strength estimator. The default is 8. This is also the minimum. For more information, see [Password settings]({{site.url}}{{site.baseurl}}/security/configuration/yaml/#password-settings).
|
||||
|
||||
- `plugins.security.restapi.password_score_based_validation_strength` (Static): Sets a threshold to determine whether the password is strong or weak. Valid values are `fair`, `good`, `strong`, and `very_strong`. This setting is used in conjunction with `plugins.security.restapi.password_min_length`.
|
||||
|
||||
- `plugins.security.unsupported.restapi.allow_securityconfig_modification` (Static): Enables the use of the PUT and PATCH methods for the configuration APIs.
|
||||
|
||||
## Advanced settings
|
||||
|
||||
The Security plugin supports the following advanced settings:
|
||||
|
||||
- `plugins.security.authcz.impersonation_dn` (Static): Enables transport layer impersonation. This allows DNs to impersonate as other users. See [User impersonation]({{site.url}}{{site.baseurl}}/security/access-control/impersonation/).
|
||||
|
||||
- `plugins.security.authcz.rest_impersonation_user` (Static): Enables REST layer impersonation. This allows users to impersonate as other users. See [User impersonation]({{site.url}}{{site.baseurl}}/security/access-control/impersonation/).
|
||||
|
||||
- `plugins.security.allow_default_init_securityindex` (Static): When set to `true`, OpenSearch Security will automatically initialize the configuration index with the files in the `/config` directory if the index does not exist.
|
||||
|
||||
This will use well-known default passwords. Use only in a private network/environment.
|
||||
{: .warning}
|
||||
|
||||
- `plugins.security.allow_unsafe_democertificates` (Static): When set to `true`, OpenSearch starts up with demo certificates. These certificates are issued only for demo purposes.
|
||||
|
||||
These certificates are well known and therefore unsafe for production. Use only in a private network/environment.
|
||||
{: .warning}
|
||||
|
||||
- `plugins.security.system_indices.permission.enabled` (Static): Enables the system index permissions feature. When set to `true`, the feature is enabled and users with permission to modify roles can create roles that include permissions that grant access to system indexes. When set to `false`, the permission is disabled and only admins with an admin certificate can make changes to system indexes. By default, the permission is set to `false` in a new cluster.
|
||||
|
||||
## Expert-level settings
|
||||
|
||||
An expert-level setting should only be configured and deployed by an admin who understands the feature completely. Misunderstandings of a feature can lead to security risks, cause the Security plugin to not operate properly, or cause data loss.
|
||||
{: .warning}
|
||||
|
||||
The Security plugin supports the following expert-level settings:
|
||||
|
||||
- `plugins.security.config_index_name` (Static): The name of the index where `.opendistro_security` stores its configuration.
|
||||
|
||||
- `plugins.security.cert.oid` (Static): Defines the Object Identifier (OID) of server node certificates.
|
||||
|
||||
- `plugins.security.cert.intercluster_request_evaluator_class` (Static): Specifies the implementation of `org.opensearch.security.transport.InterClusterRequestEvaluator` that is used to evaluate intercluster requests. Instances of `org.opensearch.security.transport.InterClusterRequestEvaluator` must implement a single-argument constructor that takes an `org.opensearch.common.settings.Settings` object.
|
||||
|
||||
- `plugins.security.enable_snapshot_restore_privilege` (Static): When set to `false`, this setting disables snapshot restore for regular users. In this case, only snapshot restore requests signed by an admin TLS certificate are accepted. When set to `true` (default), regular users can restore snapshots if they have the `cluster:admin/snapshot/restore`, `indices:admin/create`, and `indices:data/write/index` privileges.
|
||||
|
||||
A snapshot can only be restored when it does not contain global state and does not restore the `.opendistro_security` index.
|
||||
{: .note}
|
||||
|
||||
- `plugins.security.check_snapshot_restore_write_privileges` (Static): When set to `false`, additional index checks are omitted. When set to the default of `true`, attempts to restore snapshots are evaluated for `indices:admin/create` and `"indices:data/write/index`.
|
||||
|
||||
- `plugins.security.cache.ttl_minutes` (Static): Determines how long it takes for authentication caching to time out. The authentication cache helps speed up authentication by temporarily storing user objects returned from the backend so that the Security plugin is not required to make repeated requests for them. Set the value in minutes. The default is `60`. Disable caching by setting the value to `0`.
|
||||
|
||||
- `plugins.security.disabled` (Static): Disables OpenSearch Security.
|
||||
|
||||
Disabling this plugin can expose your configuration (including passwords) to the public.
|
||||
{:warning}
|
||||
|
||||
- `plugins.security.protected_indices.enabled` (Static): If set to `true`, enables protected indexes. Protected indexes are even more secure than regular indexes. These indexes require a role to access like any other traditional index and require an additional role to be visible. This setting is used in conjunction with the `plugins.security.protected_indices.roles` and `plugins.security.protected_indices.indices` settings.
|
||||
|
||||
- `plugins.security.protected_indices.roles` (Static): Specifies a list of roles to which a user must be mapped to access protected indexes.
|
||||
|
||||
- `plugins.security.protected_indices.indices` (Static): Specifies a list of indexes to mark as protected. These indexes will only be visible to users mapped to the roles specified in `plugins.security.protected_indices.roles`. After this requirement is fulfilled, a user will still need to be mapped to the traditional role used to grant access permission to the index.
|
||||
|
||||
- `plugins.security.system_indices.enabled` (Static): If set to `true`, enables system indexes. System indexes are similar to the security index, except that the contents are not encrypted. Indexes configured as system indexes can be accessed by either a super-admin or a user with a role that includes the [system index permission]({{site.url}}{{site.baseurl}}/security/access-control/permissions/#system-index-permissions). For more information about system indexes, see [System indexes]({{site.url}}{{site.baseurl}}/security/configuration/system-indices/).
|
||||
|
||||
- `plugins.security.system_indices.indices` (Static): A list of indexes to be used as system indexes. This setting is controlled by the `plugins.security.system_indices.enabled` setting.
|
||||
|
||||
- `plugins.security.allow_default_init_securityindex` (Static): When set to `true`, sets the Security plugin to its default security settings if an attempt to create the security index fails when OpenSearch launches. Default security settings are stored in YAML files contained in the `opensearch-project/security/config` directory. Default is `false`.
|
||||
|
||||
- `plugins.security.cert.intercluster_request_evaluator_class` (Static): A class to be used for evaluating intercluster communication.
|
||||
|
||||
- `plugins.security.enable_snapshot_restore_privilege` (Static): Enables granting the snapshot restore privilege. Optional. Default is `true`.
|
||||
|
||||
- `plugins.security.check_snapshot_restore_write_privileges` (Static): Enforces write privilege evaluation when creating snapshots. Default is `true`.
|
||||
|
||||
## Audit log settings
|
||||
|
||||
The Security plugin supports the following audit log settings:
|
||||
|
||||
- `plugins.security.audit.enable_rest` (Dynamic): Enables or disables REST request logging. Default is `true` (enable).
|
||||
|
||||
- `plugins.security.audit.enable_transport` (Dynamic): Enables or disables transport-level request logging. Default is `false` (disable).
|
||||
|
||||
- `plugins.security.audit.resolve_bulk_requests` (Dynamic): Enable or disable bulk request logging. When enabled, all subrequests in bulk requests are also logged. Default is `false` (disabled).
|
||||
|
||||
- `plugins.security.audit.config.disabled_categories` (Dynamic): Disables the specified event categories.
|
||||
|
||||
- `plugins.security.audit.ignore_requests` (Dynamic): Excludes the specified requests from being logged. Allows wildcards and regular expressions containing actions or REST request paths.
|
||||
|
||||
- `plugins.security.audit.threadpool.size` (Static): Determines the number of threads in the thread pool used to log events. Default is `10`. Setting this value to `0` disables the thread pool, which means the plugin logs events synchronously.
|
||||
|
||||
- `plugins.security.audit.threadpool.max_queue_len` (Static): Sets the maximum queue length per thread. Default is `100000`.
|
||||
|
||||
- `plugins.security.audit.ignore_users` (Dynamic): An array of users. Audit requests from the users in the list will not be logged.
|
||||
|
||||
- `plugins.security.audit.type` (Static): The destination of audit log events. Valid values are `internal_opensearch`, `external_opensearch`, `debug`, and `webhook`.
|
||||
|
||||
- `plugins.security.audit.config.http_endpoints` (Static): A list of endpoints for `localhost`.
|
||||
|
||||
- `plugins.security.audit.config.index` (Static): The audit log index. The default is `auditlog6`. The index can be static or an index that includes a date so that it rotates on a daily basis, for example, `"'auditlog6-'YYYY.MM.dd"`. In either case, make sure to secure the index properly.
|
||||
|
||||
- `plugins.security.audit.config.type` (Static): Specify the audit log type as `auditlog`.
|
||||
|
||||
- `plugins.security.audit.config.username` (Static): Username for the audit log configuration.
|
||||
|
||||
- `plugins.security.audit.config.password` (Static): Password for the audit log configuration.
|
||||
|
||||
- `plugins.security.audit.config.enable_ssl` (Static): Enables or disables SSL for audit logging.
|
||||
|
||||
- `plugins.security.audit.config.verify_hostnames` (Static): Enables or disables verification of the hostname for SSL/TLS certificates. Default is `true` (enabled).
|
||||
|
||||
- `plugins.security.audit.config.enable_ssl_client_auth` (Static): Enables or disables SSL/TLS client authentication. Default is `false` (disabled).
|
||||
|
||||
- `plugins.security.audit.config.cert_alias` (Static): An alias to the certificate used for audit log access.
|
||||
|
||||
- `plugins.security.audit.config.pemkey_filepath` (Static): The `/config` relative file path to the Privacy Enhanced Mail (PEM) key used for audit logging.
|
||||
|
||||
- `plugins.security.audit.config.pemkey_content` (Static): The base64-encoded content of the PEM key used for audit logging. This is an alternative to `...config.pemkey_filepath`.
|
||||
|
||||
- `plugins.security.audit.config.pemkey_password` (Static): Password for the PEM-formatted private key used by the client.
|
||||
|
||||
- `plugins.security.audit.config.pemcert_filepath` (Static): The `/config` relative file path to the PEM certificate used for audit logging.
|
||||
|
||||
- `plugins.security.audit.config.pemcert_content` (Static): The base64-encoded content of the PEM certificate used for audit logging. This is an alternative to specifying the file path with `...config.pemcert_filepath`.
|
||||
|
||||
- `plugins.security.audit.config.pemtrustedcas_filepath` (Static): The `/config` relative filepath to trusted root certificate authority.
|
||||
|
||||
- `plugins.security.audit.config.pemtrustedcas_content` (Static): The base64-encoded content of the root certificate authority. This is an alternative to `...config.pemtrustedcas_filepath`.
|
||||
|
||||
- `plugins.security.audit.config.webhook.url` (Static): The webhook URL.
|
||||
|
||||
- `plugins.security.audit.config.webhook.format` (Static): The format used for the webhook. Valid values are `URL_PARAMETER_GET`, `URL_PARAMETER_POST`, `TEXT`, `JSON`, and `SLACK`.
|
||||
|
||||
- `plugins.security.audit.config.webhook.ssl.verify` (Static): Enables or disables verification of any SSL/TLS certificates sent with any webhook request. Default is `true` (enabled).
|
||||
|
||||
- `plugins.security.audit.config.webhook.ssl.pemtrustedcas_filepath` (Static): The `/config` relative file path to trusted certificate authority against which webhook requests are verified.
|
||||
|
||||
- `plugins.security.audit.config.webhook.ssl.pemtrustedcas_content` (Static): The base64-encoded content of the certificate authority used to verify webhook requests. This is an alternative to `...config.pemtrustedcas_filepath`.
|
||||
|
||||
- `plugins.security.audit.config.log4j.logger_name` (Static): A custom name for the Log4j logger.
|
||||
|
||||
- `plugins.security.audit.config.log4j.level` (Static): Provides a default log level for the Log4j logger. Valid values are `OFF`, `FATAL`, `ERROR`, `WARN`, `INFO`, `DEBUG`, `TRACE`, and `ALL`. Default is `INFO`.
|
||||
|
||||
- `opendistro_security.audit.config.disabled_rest_categories` (Dynamic): A list of REST categories to be ignored by the logger. Valid values are `AUTHENTICATED` and `GRANTED_PRIVILEGES`.
|
||||
|
||||
- `opendistro_security.audit.config.disabled_transport_categories` (Dynamic): A list of transport layer categories to be ignored by the logger. Valid values are `AUTHENTICATED` and `GRANTED_PRIVILEGES`.
|
||||
|
||||
## Hostname verification and DNS lookup settings
|
||||
|
||||
The Security plugin supports the following hostname verification and DNS lookup settings:
|
||||
|
||||
- `plugins.security.ssl.transport.enforce_hostname_verification` (Static): Whether to verify hostnames on the transport layer. Optional. Default is `true`.
|
||||
|
||||
- `plugins.security.ssl.transport.resolve_hostname` (Static): Whether to resolve hostnames against DNS on the transport layer. Optional. Default is `true`. Only works if hostname verification is enabled.
|
||||
|
||||
For more information, see [Hostname verification and DNS lookup]({{site.url}}{{site.baseurl}}/security/configuration/tls/#advanced-hostname-verification-and-dns-lookup).
|
||||
|
||||
## Client authentication settings
|
||||
|
||||
The Security plugin supports the following client authentication setting:
|
||||
|
||||
- `plugins.security.ssl.http.clientauth_mode` (Static): The TLS client authentication mode to use. Valid values are `OPTIONAL` (default), `REQUIRE`, and `NONE`. Optional.
|
||||
|
||||
For more information, see [Client authentication]({{site.url}}{{site.baseurl}}/security/configuration/tls/#advanced-client-authentication).
|
||||
|
||||
## Enabled cipher and protocol settings
|
||||
|
||||
The Security plugin supports the following enabled cipher and protocol settings. Each setting must be expressed in an array:
|
||||
|
||||
- `plugins.security.ssl.http.enabled_ciphers` (Static): Enabled TLS cipher suites for the REST layer. Only Java format is supported.
|
||||
|
||||
- `plugins.security.ssl.http.enabled_protocols` (Static): Enabled TLS protocols for the REST layer. Only Java format is supported.
|
||||
|
||||
- `plugins.security.ssl.transport.enabled_ciphers` (Static): Enabled TLS cipher suites for the transport layer. Only Java format is supported.
|
||||
|
||||
- `plugins.security.ssl.transport.enabled_protocols` (Static): Enabled TLS protocols for the transport layer. Only Java format is supported.
|
||||
|
||||
For more information, see [Enabled ciphers and protocols]({{site.url}}{{site.baseurl}}/security/configuration/tls/#advanced-enabled-ciphers-and-protocols).
|
||||
|
||||
## Key store and trust store files---transport layer TLS settings
|
||||
|
||||
The Security plugin supports the following transport layer TLS key store and trust store settings:
|
||||
|
||||
- `plugins.security.ssl.transport.keystore_type` (Static): The type of the key store file. Optional. Valid values are `JKS` or `PKCS12/PFX`. Default is `JKS`.
|
||||
|
||||
- `plugins.security.ssl.transport.keystore_filepath` (Static): The path to the key store file, which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.transport.keystore_alias` (Static): The key store alias name. Optional. Default is the first alias.
|
||||
|
||||
- `plugins.security.ssl.transport.keystore_password` (Static): The key store password. Default is `changeit`.
|
||||
|
||||
- `plugins.security.ssl.transport.truststore_type` (Static): The type of the trust store file. Optional. Valid values are `JKS` or `PKCS12/PFX`. Default is `JKS`.
|
||||
|
||||
- `plugins.security.ssl.transport.truststore_filepath` (Static): The path to the trust store file, which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.transport.truststore_alias` (Static): The trust store alias name. Optional. Default is all certificates.
|
||||
|
||||
- `plugins.security.ssl.transport.truststore_password` (Static): The trust store password. Default is `changeit`.
|
||||
|
||||
For more information about key store and trust store files, see [Transport layer TLS]({{site.url}}{{site.baseurl}}/security/configuration/tls/#transport-layer-tls-1).
|
||||
|
||||
## Key store and trust store files---REST layer TLS settings
|
||||
|
||||
The Security plugin supports the following REST layer TLS key store and trust store settings:
|
||||
|
||||
- `plugins.security.ssl.http.enabled` (Static): Whether to enable TLS on the REST layer. If enabled, only HTTPS is allowed. Optional. Default is `false`.
|
||||
|
||||
- `plugins.security.ssl.http.keystore_type` (Static): The type of the key store file. Optional. Valid values are `JKS` or `PKCS12/PFX`. Default is `JKS`.
|
||||
|
||||
- `plugins.security.ssl.http.keystore_filepath` (Static): The path to the key store file, which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.http.keystore_alias` (Static): The key store alias name. Optional. Default is the first alias.
|
||||
|
||||
- `plugins.security.ssl.http.keystore_password`: The key store password. Default is `changeit`.
|
||||
|
||||
- `plugins.security.ssl.http.truststore_type`: The type of the trust store file. Optional. Valid values are `JKS` or `PKCS12/PFX`. Default is `JKS`.
|
||||
|
||||
- `plugins.security.ssl.http.truststore_filepath`: The path to the trust store file, which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.http.truststore_alias` (Static): The trust store alias name. Optional. Default is all certificates.
|
||||
|
||||
- `plugins.security.ssl.http.truststore_password` (Static): The trust store password. Default is `changeit`.
|
||||
|
||||
For more information, see [REST layer TLS]({{site.url}}{{site.baseurl}}/security/configuration/tls/#rest-layer-tls-1).
|
||||
|
||||
## OpenSSL settings
|
||||
|
||||
The Security plugin supports the following OpenSSL settings:
|
||||
|
||||
- `plugins.security.ssl.transport.enable_openssl_if_available` (Static): Enables OpenSSL on the transport layer if available. Optional. Default is `true`.
|
||||
|
||||
- `plugins.security.ssl.http.enable_openssl_if_available` (Static): Enables OpenSSL on the REST layer if available. Optional. Default is `true`.
|
||||
|
||||
For more information, see [OpenSSL]({{site.url}}{{site.baseurl}}/security/configuration/tls/#advanced-openssl).
|
||||
|
||||
## X.509 PEM certificates and PKCS #8 keys---transport layer TLS settings
|
||||
|
||||
The Security plugin supports the following transport layer TLS settings related to X.509 PEM certificates and PKCS #8 keys:
|
||||
|
||||
- `plugins.security.ssl.transport.pemkey_filepath` (Static): The path to the certificate's key file (PKCS #8), which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.transport.pemkey_password` (Static): The key password. Omit this setting if the key has no password. Optional.
|
||||
|
||||
- `plugins.security.ssl.transport.pemcert_filepath` (Static): The path to the X.509 node certificate chain (PEM format), which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.transport.pemtrustedcas_filepath` (Static): The path to the root certificate authorities (PEM format), which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
For more information, see [REST layer TLS]({{site.url}}{{site.baseurl}}/security/configuration/tls/#transport-layer-tls).
|
||||
|
||||
## X.509 PEM certificates and PKCS #8 keys---REST layer TLS settings
|
||||
|
||||
The Security plugin supports the following REST layer TLS settings related to X.509 PEM certificates and PKCS #8 keys:
|
||||
|
||||
- `plugins.security.ssl.http.enabled` (Static): Whether to enable TLS on the REST layer. If enabled, only HTTPS is allowed. Optional. Default is `false`.
|
||||
|
||||
- `plugins.security.ssl.http.pemkey_filepath` (Static): The path to the certificate’s key file (PKCS #8), which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.http.pemkey_password` (Static): The key password. Omit this setting if the key has no password. Optional.
|
||||
|
||||
- `plugins.security.ssl.http.pemcert_filepath` (Static): The path to the X.509 node certificate chain (PEM format), which must be under the `config` directory, specified using a relative path. Required.
|
||||
|
||||
- `plugins.security.ssl.http.pemtrustedcas_filepath`: The path to the root certificate authorities (PEM format), which must be under the config directory, specified using a relative path. Required.
|
||||
|
||||
For more information, see [REST layer TLS]({{site.url}}{{site.baseurl}}/security/configuration/tls/#rest-layer-tls).
|
||||
|
||||
## Transport layer security settings
|
||||
|
||||
The Security plugin supports the following transport layer security settings:
|
||||
|
||||
- `plugins.security.ssl.transport.enabled` (Static): Whether to enable TLS on the REST layer.
|
||||
|
||||
- `plugins.security.ssl.transport.client.pemkey_password` (Static): The password for the PEM-formatted private key used by the transport client.
|
||||
|
||||
- `plugins.security.ssl.transport.keystore_keypassword` (Static): The password for the key inside the key store.
|
||||
|
||||
- `plugins.security.ssl.transport.server.keystore_keypassword` (Static): The password for the key inside the server key store.
|
||||
|
||||
- `plugins.sercurity.ssl.transport.server.keystore_alias` (Static): The alias name for the key store of the server.
|
||||
|
||||
- `plugins.sercurity.ssl.transport.client.keystore_alias` (Static): The alias name for the key store of the client.
|
||||
|
||||
- `plugins.sercurity.ssl.transport.server.truststore_alias` (Static): The alias name for the trust store of the server.
|
||||
|
||||
- `plugins.sercurity.ssl.transport.client.truststore_alias` (Static): The alias name for the trust store of the client.
|
||||
|
||||
- `plugins.security.ssl.client.external_context_id` (Static): Provides the transport client an ID to use for an external SSL context.
|
||||
|
||||
- `plugins.secuirty.ssl.transport.principal_extractor_class` (Static): Specifies a class implementing an extractor so a custom part of the certificate is used as the principal.
|
||||
|
||||
- `plugins.security.ssl.http.crl.file_path` (Static): A file path to a certificate revocation list file.
|
||||
|
||||
- `plugins.security.ssl.http.crl.validate` (Static): Enables certificate revocation list (CRL) validation. Default is `false` (disabled).
|
||||
|
||||
- `plugins.security.ssl.http.crl.prefer_crlfile_over_ocsp` (Static): Whether to prefer the CRL certificate entry over the Online Certificate Status Protocol (OCSP) entry if the certificate contains both. Optional. Default is `false`.
|
||||
|
||||
- `plugins.security.ssl.http.crl.check_only_end_entitites` (Static): When `true`, only leaf certificates are validated. Default is `true`.
|
||||
|
||||
- `plugins.security.ssl.http.crl.disable_ocsp` (Static): Disables OCSP. Default is `false` (OCSP is enabled).
|
||||
|
||||
- `plugins.security.ssl.http.crl.disable_crldp` (Static): Disables CRL endpoints in certificates. Default is `false` (CRL endpoints are enabled).
|
||||
|
||||
- `plugins.security.ssl.allow_client_initiated_renegotiation` (Static): Enables or disables client renegotiation. Default is `false` (client initiated renegotiation is not allowed).
|
||||
|
||||
## Security plugin settings examples
|
||||
|
||||
```yml
|
||||
# Common configuration settings
|
||||
plugins.security.nodes_dn:
|
||||
- "CN=*.example.com, OU=SSL, O=Test, L=Test, C=DE"
|
||||
- "CN=node.other.com, OU=SSL, O=Test, L=Test, C=DE"
|
||||
plugins.security.authcz.admin_dn:
|
||||
- CN=kirk,OU=client,O=client,L=test, C=de
|
||||
plugins.security.roles_mapping_resolution: MAPPING_ONLY
|
||||
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
|
||||
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
|
||||
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
|
||||
plugins.security.ssl.transport.enforce_hostname_verification: false
|
||||
plugins.security.ssl.http.enabled: true
|
||||
plugins.security.ssl.http.pemcert_filepath: esnode.pem
|
||||
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
|
||||
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
|
||||
plugins.security.allow_unsafe_democertificates: true
|
||||
plugins.security.allow_default_init_securityindex: true
|
||||
plugins.security.nodes_dn_dynamic_config_enabled: false
|
||||
plugins.security.cert.intercluster_request_evaluator_class: # need example value for this.
|
||||
plugins.security.audit.type: internal_opensearch
|
||||
plugins.security.enable_snapshot_restore_privilege: true
|
||||
plugins.security.check_snapshot_restore_write_privileges: true
|
||||
plugins.security.cache.ttl_minutes: 60
|
||||
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
|
||||
plugins.security.system_indices.enabled: true
|
||||
plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*"]
|
||||
node.max_local_storage_nodes: 3
|
||||
plugins.security.restapi.password_validation_regex: '(?=.*[A-Z])(?=.*[^a-zA-Z\d])(?=.*[0-9])(?=.*[a-z]).{8,}'
|
||||
plugins.security.restapi.password_validation_error_message: "Password must be minimum 8 characters long and must contain at least one uppercase letter, one lowercase letter, one digit, and one special character."
|
||||
plugins.security.allow_default_init_securityindex: true
|
||||
plugins.security.cache.ttl_minutes: 60
|
||||
#
|
||||
# REST Management API configuration settings
|
||||
plugins.security.restapi.roles_enabled: ["all_access","xyz_role"]
|
||||
plugins.security.restapi.endpoints_disabled.all_access.ACTIONGROUPS: ["PUT","POST","DELETE"] # Alternative example: plugins.security.restapi.endpoints_disabled.xyz_role.LICENSE: ["DELETE"] #
|
||||
# Audit log configuration settings
|
||||
plugins.security.audit.enable_rest: true
|
||||
plugins.security.audit.enable_transport: false
|
||||
plugins.security.audit.resolve_bulk_requests: false
|
||||
plugins.security.audit.config.disabled_categories: ["AUTHENTICATED","GRANTED_PRIVILEGES"]
|
||||
plugins.security.audit.ignore_requests: ["indices:data/read/*","*_bulk"]
|
||||
plugins.security.audit.threadpool.size: 10
|
||||
plugins.security.audit.threadpool.max_queue_len: 100000
|
||||
plugins.security.audit.ignore_users: ['kibanaserver','some*user','/also.*regex possible/']
|
||||
plugins.security.audit.type: internal_opensearch
|
||||
#
|
||||
# external_opensearch settings
|
||||
plugins.security.audit.config.http_endpoints: ['localhost:9200','localhost:9201','localhost:9202']
|
||||
plugins.security.audit.config.index: "'auditlog6-'2023.06.15"
|
||||
plugins.security.audit.config.type: auditlog
|
||||
plugins.security.audit.config.username: auditloguser
|
||||
plugins.security.audit.config.password: auditlogpassword
|
||||
plugins.security.audit.config.enable_ssl: false
|
||||
plugins.security.audit.config.verify_hostnames: false
|
||||
plugins.security.audit.config.enable_ssl_client_auth: false
|
||||
plugins.security.audit.config.cert_alias: mycert
|
||||
plugins.security.audit.config.pemkey_filepath: key.pem
|
||||
plugins.security.audit.config.pemkey_content: <...pem base 64 content>
|
||||
plugins.security.audit.config.pemkey_password: secret
|
||||
plugins.security.audit.config.pemcert_filepath: cert.pem
|
||||
plugins.security.audit.config.pemcert_content: <...pem base 64 content>
|
||||
plugins.security.audit.config.pemtrustedcas_filepath: ca.pem
|
||||
plugins.security.audit.config.pemtrustedcas_content: <...pem base 64 content>
|
||||
#
|
||||
# Webhook settings
|
||||
plugins.security.audit.config.webhook.url: "http://mywebhook/endpoint"
|
||||
plugins.security.audit.config.webhook.format: JSON
|
||||
plugins.security.audit.config.webhook.ssl.verify: false
|
||||
plugins.security.audit.config.webhook.ssl.pemtrustedcas_filepath: ca.pem
|
||||
plugins.security.audit.config.webhook.ssl.pemtrustedcas_content: <...pem base 64 content>
|
||||
#
|
||||
# log4j settings
|
||||
plugins.security.audit.config.log4j.logger_name: auditlogger
|
||||
plugins.security.audit.config.log4j.level: INFO
|
||||
#
|
||||
# Advanced configuration settings
|
||||
plugins.security.authcz.impersonation_dn:
|
||||
"CN=spock,OU=client,O=client,L=Test,C=DE":
|
||||
- worf
|
||||
"cn=webuser,ou=IT,ou=IT,dc=company,dc=com":
|
||||
- user2
|
||||
- user1
|
||||
plugins.security.authcz.rest_impersonation_user:
|
||||
"picard":
|
||||
- worf
|
||||
"john":
|
||||
- steve
|
||||
- martin
|
||||
plugins.security.allow_default_init_securityindex: false
|
||||
plugins.security.allow_unsafe_democertificates: false
|
||||
plugins.security.cache.ttl_minutes: 60
|
||||
plugins.security.restapi.password_validation_regex: '(?=.*[A-Z])(?=.*[^a-zA-Z\d])(?=.*[0-9])(?=.*[a-z]).{8,}'
|
||||
plugins.security.restapi.password_validation_error_message: "A password must be at least 8 characters long and contain at least one uppercase letter, one lowercase letter, one digit, and one special character."
|
||||
plugins.security.restapi.password_min_length: 8
|
||||
plugins.security.restapi.password_score_based_validation_strength: very_strong
|
||||
#
|
||||
# Advanced SSL settings - use only if you understand SSL ins and outs
|
||||
plugins.security.ssl.transport.client.pemkey_password: superSecurePassword1
|
||||
plugins.security.ssl.transport.keystore_keypassword: superSecurePassword2
|
||||
plugins.security.ssl.transport.server.keystore_keypassword: superSecurePassword3
|
||||
plugins.security.ssl.http.keystore_keypassword: superSecurePassword4
|
||||
plugins.security.ssl.http.clientauth_mode: REQUIRE
|
||||
plugins.security.ssl.transport.enabled: true
|
||||
plugins.security.ssl.transport.server.keystore_alias: my_alias
|
||||
plugins.security.ssl.transport.client.keystore_alias: my_other_alias
|
||||
plugins.security.ssl.transport.server.truststore_alias: trustore_alias_1
|
||||
plugins.security.ssl.transport.client.truststore_alias: trustore_alias_2
|
||||
plugins.security.ssl.client.external_context_id: my_context_id
|
||||
plugins.security.ssl.transport.principal_extractor_class: org.opensearch.security.ssl.ExampleExtractor
|
||||
plugins.security.ssl.http.crl.file_path: ssl/crl/revoked.crl
|
||||
plugins.security.ssl.http.crl.validate: true
|
||||
plugins.security.ssl.http.crl.prefer_crlfile_over_ocsp: true
|
||||
plugins.security.ssl.http.crl.check_only_end_entitites: false
|
||||
plugins.security.ssl.http.crl.disable_ocsp: true
|
||||
plugins.security.ssl.http.crl.disable_crldp: true
|
||||
plugins.security.ssl.allow_client_initiated_renegotiation: true
|
||||
#
|
||||
# Expert settings - use only if you understand their use completely: accidental values can potentially cause security risks or failures to OpenSearch Security.
|
||||
plugins.security.config_index_name: .opendistro_security
|
||||
plugins.security.cert.oid: '1.2.3.4.5.5'
|
||||
plugins.security.cert.intercluster_request_evaluator_class: org.opensearch.security.transport.DefaultInterClusterRequestEvaluator
|
||||
plugins.security.enable_snapshot_restore_privilege: true
|
||||
plugins.security.check_snapshot_restore_write_privileges: true
|
||||
plugins.security.cache.ttl_minutes: 60
|
||||
plugins.security.disabled: false
|
||||
plugins.security.protected_indices.enabled: true
|
||||
plugins.security.protected_indices.roles: ['all_access']
|
||||
plugins.security.protected_indices.indices: []
|
||||
plugins.security.system_indices.enabled: true
|
||||
plugins.security.system_indices.indices: ['.opendistro-alerting-config', '.opendistro-ism-*', '.opendistro-reports-*', '.opensearch-notifications-*', '.opensearch-notebooks', '.opensearch-observability', '.opendistro-asynchronous-search-response*', '.replication-metadata-store']
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
|
@ -23,7 +23,7 @@ OpenSearch and OpenSearch Dashboards are available on any compatible host that s
|
|||
| [Ansible playbook]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/ansible/) | |
|
||||
| [Windows]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/windows/) | [Windows]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/windows/) |
|
||||
|
||||
After you've installed OpenSearch, learn about [configuring]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/) it for your deployment.
|
||||
After you've installed OpenSearch, learn about [configuring]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) it for your deployment.
|
||||
|
||||
For more information about upgrading your OpenSearch cluster, see the [upgrade guide]({{site.url}}{{site.baseurl}}/install-and-configure/upgrade-opensearch/index/).
|
||||
|
||||
|
|
|
|||
|
|
@ -240,7 +240,7 @@ The following recommended settings will allow you to:
|
|||
- Configure your own TLS certificates—no third-party certificate authority (CA) is required.
|
||||
- Create an admin user with a custom password.
|
||||
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/) for guidance before proceeding.
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) for guidance before proceeding.
|
||||
{:.note}
|
||||
|
||||
Before modifying any configuration files, it's always a good idea to save a backup copy before making changes. The backup file can be used to mitigate any issues caused by a bad configuration.
|
||||
|
|
@ -535,7 +535,7 @@ sudo apt-get upgrade opensearch=<version>
|
|||
|
||||
## Related links
|
||||
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/)
|
||||
- [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/)
|
||||
- [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
|
||||
- [About the Security plugin]({{site.url}}{{site.baseurl}}/security-plugin/index/)
|
||||
|
|
|
|||
|
|
@ -486,7 +486,7 @@ COPY --chown=opensearch:opensearch my-root-cas.pem /usr/share/opensearch/config/
|
|||
|
||||
## Related links
|
||||
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/)
|
||||
- [Performance analyzer]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/index/)
|
||||
- [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/)
|
||||
- [About Security in OpenSearch]({{site.url}}{{site.baseurl}}/security/index/)
|
||||
|
|
|
|||
|
|
@ -209,7 +209,7 @@ The following recommended settings will allow you to:
|
|||
- Configure your own TLS certificates—no third-party certificate authority (CA) is required.
|
||||
- Create an admin user with a custom password.
|
||||
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/) for guidance before proceeding.
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) for guidance before proceeding.
|
||||
{:.note}
|
||||
|
||||
Before modifying any configuration files, it's always a good idea to save a backup copy before making changes. The backup file can be used to mitigate any issues caused by a bad configuration.
|
||||
|
|
@ -502,7 +502,7 @@ sudo yum update
|
|||
|
||||
## Related links
|
||||
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/)
|
||||
- [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/)
|
||||
- [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
|
||||
- [About the Security plugin]({{site.url}}{{site.baseurl}}/security/index/)
|
||||
|
|
@ -234,7 +234,7 @@ The following recommended settings will allow you to:
|
|||
- Configure your own TLS certificates - no third-party certificate authority (CA) is required.
|
||||
- Create an admin user with a custom password.
|
||||
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/) for guidance before proceeding.
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) for guidance before proceeding.
|
||||
{:.note}
|
||||
|
||||
Before modifying any configuration files, it's always a good idea to save a backup copy before making changes. The backup file can be used to revert any issues caused by a bad configuration.
|
||||
|
|
@ -586,7 +586,7 @@ The following configuration is only suitable for testing in a non-production env
|
|||
|
||||
## Related links
|
||||
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/)
|
||||
- [Configure Performance Analyzer for Tarball Installation]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/index/#install-performance-analyzer)
|
||||
- [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/)
|
||||
- [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
|
||||
|
|
|
|||
|
|
@ -205,7 +205,7 @@ The following recommended settings will allow you to:
|
|||
- Set initial and maximum JVM heap sizes.
|
||||
- Define an environment variable that points to the bundled JDK.
|
||||
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/) for guidance before proceeding.
|
||||
If you ran the security demo script, then you will need to manually reconfigure settings that were modified. Refer to [Security configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) for guidance before proceeding.
|
||||
{:.note}
|
||||
|
||||
Before modifying any configuration files, it's always a good idea to save a backup copy before making changes. The backup file can be used to revert any issues caused by a bad configuration.
|
||||
|
|
@ -259,6 +259,6 @@ The Performance Analyzer plugin is not available on Windows. All other OpenSearc
|
|||
|
||||
## Related links
|
||||
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/)
|
||||
- [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
|
||||
- [About the Security plugin]({{site.url}}{{site.baseurl}}/security/index/)
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
layout: default
|
||||
title: Upgrading OpenSearch
|
||||
nav_order: 4
|
||||
nav_order: 20
|
||||
has_children: true
|
||||
redirect_from:
|
||||
- /upgrade-opensearch/index/
|
||||
|
|
|
|||
|
|
@ -197,7 +197,7 @@ Review [Upgrading OpenSearch]({{site.url}}{{site.baseurl}}/upgrade-opensearch/in
|
|||
|
||||
### Related articles
|
||||
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/)
|
||||
- [Performance analyzer]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/index/)
|
||||
- [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/)
|
||||
- [About Security in OpenSearch]({{site.url}}{{site.baseurl}}/security/index/)
|
||||
|
|
|
|||
|
|
@ -9,6 +9,8 @@ nav_order: 10
|
|||
|
||||
To enhance and customize your OpenSearch cluster for machine learning (ML), you can add and modify several configuration settings for the ML Commons plugin in your 'opensearch.yml' file.
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## ML node
|
||||
|
||||
By default, ML tasks and models only run on ML nodes. When configured without the `data` node role, ML nodes do not store any shards and instead calculate resource requirements at runtime. To use an ML node, create a node in your `opensearch.yml` file. Give your node a custom name and define the node role as `ml`:
|
||||
|
|
|
|||
|
|
@ -130,3 +130,16 @@ The following table describes the request parameters configured in the previous
|
|||
The logic used by your job should be defined by a class extended from `ScheduledJobRunner` in the `SampleJobParameter.java` sample file, such as `SampleJobRunner`. While the job is running, there is a locking mechanism you can use to prevent other nodes from running the same job. First, [acquire](https://github.com/opensearch-project/job-scheduler/blob/main/sample-extension-plugin/src/main/java/org/opensearch/jobscheduler/sampleextension/SampleJobRunner.java#L96) the lock. Then make sure to release the lock before the [job finishes](https://github.com/opensearch-project/job-scheduler/blob/main/sample-extension-plugin/src/main/java/org/opensearch/jobscheduler/sampleextension/SampleJobRunner.java#L116).
|
||||
|
||||
For more information, see the Job Scheduler [sample extension](https://github.com/opensearch-project/job-scheduler/blob/main/sample-extension-plugin/src/main/java/org/opensearch/jobscheduler/sampleextension/SampleJobParameter.java) directory in the [Job Scheduler GitHub repo](https://github.com/opensearch-project/job-scheduler).
|
||||
|
||||
## Job Scheduler cluster settings
|
||||
|
||||
The Job Scheduler plugin supports the following cluster settings. All settings are dynamic. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
| Setting | Data type | Description |
|
||||
:--- | :--- | :---
|
||||
| `plugins.jobscheduler.jitter_limit` | Double | Defines the maximum delay multiplier for job execution time. Too many jobs starting at the same time can cause high resource consumption. To balance the load, you can add a random jitter delay to the start time. For example, if the time interval is 10 minutes and the jitter is 0.6, the next job run will be randomly delayed by a time period between 0 and 6 minutes. |
|
||||
| `plugins.jobscheduler.request_timeout` | Time unit | The background sweep search timeout. Background sweep refers to the automatic scheduling and execution of registered jobs. It occurs on an interval and iterates through each extending plugin's registered job index, searching for jobs to be executed. |
|
||||
| `plugins.jobscheduler.retry_count` | Integer | Used to define the retry count of an exponential backoff policy. Backoff policies determine how long bulk processors will wait before the bulk operation is retried. It is used whenever bulk indexing requests are impacted or rejected because of resource constraints at the time of a request. For the Job Scheduler plugin, this impacts searching registered job indexes. |
|
||||
| `plugins.jobscheduler.sweeper.backoff_millis` | Time unit | Used to define the initial wait period of an exponential backoff policy, in milliseconds. Backoff policies determine how long bulk processors will wait before the bulk operation is retried. It is used whenever bulk indexing requests are impacted or rejected because of resource constraints at the time of a request. For the Job Scheduler plugin, this impacts searching registered job indexes. |
|
||||
| `plugins.jobscheduler.sweeper.page_size` | Integer | Configures the search request used to find job documents within a registered job index. Defines the number of search hits to return. |
|
||||
| `plugins.jobscheduler.sweeper.period` | Time unit | Defines the initial delay period before a background sweep is executed. |
|
||||
|
|
|
|||
|
|
@ -7,10 +7,11 @@ redirect_from:
|
|||
- /monitoring-plugins/ad/settings/
|
||||
---
|
||||
|
||||
# Settings
|
||||
# Anomaly Detection settings
|
||||
|
||||
The Anomaly Detection plugin adds several settings to the standard OpenSearch cluster settings.
|
||||
The settings are dynamic, so you can change the default behavior of the plugin without restarting your cluster. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
The anomaly detection plugin adds several settings to the standard OpenSearch cluster settings.
|
||||
The settings are dynamic, so you can change the default behavior of the plugin without restarting your cluster.
|
||||
You can mark settings as `persistent` or `transient`.
|
||||
|
||||
For example, to update the retention period of the result index:
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ GET _cat/indices?expand_wildcards=open,hidden
|
|||
|
||||
We don't recommend changing these settings; the defaults should work well for most use cases.
|
||||
|
||||
All settings are available using the OpenSearch `_cluster/settings` API. None require a restart, and all can be marked `persistent` or `transient`.
|
||||
All settings are available using the OpenSearch `_cluster/settings` API. None require a restart, and all can be marked `persistent` or `transient`. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
Setting | Default | Description
|
||||
:--- | :--- | :---
|
||||
|
|
|
|||
|
|
@ -5,9 +5,11 @@ parent: Asynchronous search
|
|||
nav_order: 4
|
||||
---
|
||||
|
||||
# Settings
|
||||
# Asynchronous Search settings
|
||||
|
||||
The Asynchronous Search plugin adds several settings to the standard OpenSearch cluster settings. They are dynamic, so you can change the default behavior of the plugin without restarting your cluster. You can mark the settings as `persistent` or `transient`.
|
||||
The Asynchronous Search plugin adds several settings to the standard OpenSearch cluster settings. They are dynamic, so you can change the default behavior of the plugin without restarting your cluster. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
You can mark the settings as `persistent` or `transient`.
|
||||
|
||||
For example, to update the retention period of the result index:
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ nav_order: 40
|
|||
|
||||
# k-NN settings
|
||||
|
||||
The k-NN plugin adds several new cluster settings.
|
||||
The k-NN plugin adds several new cluster settings. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
## Cluster settings
|
||||
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ redirect_from:
|
|||
|
||||
# SQL settings
|
||||
|
||||
The SQL plugin adds a few settings to the standard OpenSearch cluster settings. Most are dynamic, so you can change the default behavior of the plugin without restarting your cluster.
|
||||
The SQL plugin adds a few settings to the standard OpenSearch cluster settings. Most are dynamic, so you can change the default behavior of the plugin without restarting your cluster. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
It is possible to independently disable processing of `PPL` or `SQL` queries.
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
layout: default
|
||||
title: Security Analytics settings
|
||||
nav_order: 100
|
||||
has_children: false
|
||||
---
|
||||
|
||||
# Security Analytics settings
|
||||
|
||||
The Security Analytics plugin supports the following settings. All settings in this list are dynamic:
|
||||
|
||||
`plugins.security_analytics.index_timeout` (Time value): The timeout for creating detectors, findings, rules, and custom log types using the REST APIs. Default is 60 seconds.
|
||||
|
||||
`plugins.security_analytics.alert_history_enabled` (Boolean): Specifies whether to create `.opensearch-sap-<detector_type>-alerts-history-<date>` indexes. Default is `true`.
|
||||
|
||||
`plugins.security_analytics.alert_finding_enabled` (Boolean): Specifies whether to create `.opensearch-sap-<detector_type>-findings-<date>` indexes. Default is `true`.
|
||||
|
||||
`plugins.security_analytics.alert_history_rollover_period` (Time value): Specifies how frequently to roll over and delete alert history indexes. Default is 12 hours.
|
||||
|
||||
`plugins.security_analytics.alert_finding_rollover_period` (Time value): Specifies how frequently to roll over and delete finding history indexes. Default is 12 hours.
|
||||
|
||||
`plugins.security_analytics.correlation_history_rollover_period` (Time value): Specifies how frequently to roll over and delete correlation history indexes. Default is 12 hours.
|
||||
|
||||
`plugins.security_analytics.alert_history_max_age` (Time value): The oldest document to store in the alert history index before creating a new index. If the number of alerts in this time period does not exceed `alert_history_max_docs`, a new alert history index is created per period (for example, one index every 30 days). Default is 30 days.
|
||||
|
||||
`plugins.security_analytics.finding_history_max_age` (Time value): The oldest document to store in the finding history index before creating a new index. If the number of findings in this time period does not exceed `finding_history_max_docs`, a new finding history index is created per period (for example, one index every 30 days). Default is 30 days.
|
||||
|
||||
`plugins.security_analytics.correlation_history_max_age` (Time value): The oldest document to store in the correlation history index before creating a new index. If the number of correlations in this time period does not exceed `correlation_history_max_docs`, a new correlation history index is created per period (for example, one index every 30 days). Default is 30 days.
|
||||
|
||||
`plugins.security_analytics.alert_history_max_docs` (Integer): The maximum number of alerts to store in the alert history index before creating a new index. Default is 1,000.
|
||||
|
||||
`plugins.security_analytics.alert_finding_max_docs` (Integer): The maximum number of findings to store in the findings history index before creating a new index. Default is 1,000.
|
||||
|
||||
`plugins.security_analytics.correlation_history_max_docs` (Integer): The maximum number of correlations to store in the correlation history index before creating a new index. Default is 1,000.
|
||||
|
||||
`plugins.security_analytics.alert_history_retention_period` (Time value): The amount of time to keep alert history indexes before automatically deleting them. Default is 60 days.
|
||||
|
||||
`plugins.security_analytics.finding_history_retention_period` (Time value): The amount of time to keep finding history indexes before automatically deleting them. Default is 60 days.
|
||||
|
||||
`plugins.security_analytics.correlation_history_retention_period` (Time value): The amount of time to keep correlation history indexes before automatically deleting them. Default is 60 days.
|
||||
|
||||
`plugins.security_analytics.request_timeout` (Time value): The timeout for all requests the Security Analytics plugin sends to other parts of OpenSearch. Default is 10 seconds.
|
||||
|
||||
`plugins.security_analytics.action_throttle_max_value` (Time value): The maximum amount of time you can set for action throttling. Default is 24 hours. (This value displays as 1440 minutes in OpenSearch Dashboards.)
|
||||
|
||||
`plugins.security_analytics.filter_by_backend_roles` (Boolean): When set to `true`, restricts access to detectors, alerts, findings, and custom log types by backend role when enabled. Default is `false`.
|
||||
|
||||
`plugins.security_analytics.enable_workflow_usage` (Boolean): Supports the Alerting plugin workflow integration with Security Analytics. Determines whether composite monitor workflows are generated for the Alerting plugin after creating a new threat detector in Security Analytics. When set to `true`, composite monitor workflows based on an associated threat detector's configuration are enabled. When set to `false`, composite monitor workflows based on an associated threat detector's configuration are disabled. Default is `true`. For more information about Alerting plugin workflow integration with Security Analytics, see [Integrated Alerting plugin workflows]({{site.url}}{{site.baseurl}}/security-analytics/sec-analytics-config/detectors-config/#integrated-alerting-plugin-workflows).
|
||||
|
||||
`plugins.security_analytics.correlation_time_window` (Time value): Security Analytics generates correlations within a time window. This setting specifies the time window within which documents must be indexed into the index in order to be included in the same correlation. Default is 5 minutes.
|
||||
|
||||
`plugins.security_analytics.mappings.default_schema` (String): The default mapping schema used for configuring a field mapping for a security analytics detector. Default is `ecs`.
|
||||
|
||||
`plugins.security_analytics.threatintel.tifjob.update_interval` (Time value): The threat intelligence feature uses a job runner to periodically fetch new feeds. This setting is the rate at which the runner fetches and updates these new feeds. Default is 1440 minutes.
|
||||
|
||||
`plugins.security_analytics.threatintel.tifjob.batch_size` (Integer): The maximum number of documents to ingest in a bulk request during the threat intelligence feed data creation process. Default is 10,000.
|
||||
|
||||
`plugins.security_analytics.threat_intel_timeout` (Time value): The timeout value for creating and deleting threat intelligence feed data. Default is 30 seconds.
|
||||
|
||||
To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
|
@ -26,3 +26,6 @@ If you don't want to use the plugin, see [Disable security]({{site.url}}{{site.b
|
|||
|
||||
The Security plugin has several default users, roles, action groups, permissions, and settings for OpenSearch Dashboards that use kibana in their names. We will change these names in a future release.
|
||||
{: .note }
|
||||
|
||||
For a full list of `opensearch.yml` Security plugin settings, Security plugin settings, see [Security settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/security-settings/).
|
||||
{: .note}
|
||||
|
|
@ -120,6 +120,9 @@ plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opend
|
|||
node.max_local_storage_nodes: 3
|
||||
```
|
||||
|
||||
For a full list of `opensearch.yml` Security plugin settings, see [Security settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/security-settings/).
|
||||
{: .note}
|
||||
|
||||
### Refining your configuration
|
||||
|
||||
The `plugins.security.allow_default_init_securityindex` setting, when set to `true`, sets the Security plugin to its default security settings if an attempt to create the security index fails when OpenSearch launches. Default security settings are stored in YAML files contained in the `opensearch-project/security/config` directory. By default, this setting is `false`.
|
||||
|
|
|
|||
|
|
@ -65,7 +65,7 @@ OpenSearch includes other features that complement the security infrastructure.
|
|||
### Dashboards multi-tenancy
|
||||
|
||||
One such feature is OpenSearch Dashboards multi-tenancy. Tenants are work spaces that include visualizations, index patterns, and other Dashboards objects. Multi-tenancy allows for the sharing of tenants among users of Dashboards and leverages OpenSearch roles to manage access to tenants and safely make them available to others.
|
||||
For more information on creating tenants, see [OpenSearch Dashboards multi-tenancy]({{site.url}}{{site.baseurl}}/security/multi-tenancy/tenant-index/).
|
||||
For more information about creating tenants, see [OpenSearch Dashboards multi-tenancy]({{site.url}}{{site.baseurl}}/security/multi-tenancy/tenant-index/).
|
||||
|
||||
### Cross-cluster search
|
||||
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ PUT _cluster/settings
|
|||
}
|
||||
```
|
||||
|
||||
[Just like any other setting]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/), the alternative is to add the following line to `opensearch.yml` on each node and then restart the node:
|
||||
[Just like any other setting]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/), the alternative is to add the following line to `opensearch.yml` on each node and then restart the node:
|
||||
|
||||
```yml
|
||||
compatibility.override_main_response_version: true
|
||||
|
|
|
|||
|
|
@ -10,7 +10,8 @@ redirect_from:
|
|||
# Replication settings
|
||||
|
||||
The replication plugin adds several settings to the standard OpenSearch cluster settings.
|
||||
The settings are dynamic, so you can change the default behavior of the plugin without restarting your cluster.
|
||||
The settings are dynamic, so you can change the default behavior of the plugin without restarting your cluster. To learn more about static and dynamic settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/).
|
||||
|
||||
You can mark settings as `persistent` or `transient`.
|
||||
|
||||
For example, to update how often the follower cluster polls the leader cluster for updates:
|
||||
|
|
|
|||
|
|
@ -130,7 +130,7 @@ Create an index and define field mappings using a dataset provided by the OpenSe
|
|||
|
||||
You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards and added some sample data. Now you're ready to learn about configuration and functionality in more detail. Here are a few recommendations on where to begin:
|
||||
- [About the Security plugin]({{site.url}}{{site.baseurl}}/security/index/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/)
|
||||
- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/)
|
||||
- [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
|
||||
- [Getting started with OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/)
|
||||
- [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/index/)
|
||||
|
|
|
|||
Loading…
Reference in New Issue