honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/docs/reference/migration/migrate_5_0/settings.asciidoc

240 lines
10 KiB
Plaintext
Raw Normal View History

Reworked 5.0 breaking changes docs
2016-03-13 16:17:48 -04:00
[[breaking_50_settings_changes]]
=== Settings changes
From Elasticsearch 5.0 on all settings are validated before they are applied.
Node level and default index level settings are validated on node startup,
dynamic cluster and index setting are validated before they are updated/added
to the cluster state.
Every setting must be a *known* setting. All settings must have been
registered with the node or transport client they are used with. This implies
that plugins that define custom settings must register all of their settings
during plugin loading using the `SettingsModule#registerSettings(Setting)`
method.
Improve upgrade experience of node level index settings In 5.0 we don't allow index settings to be specified on the node level ie. in yaml files or via commandline argument. This can cause problems during upgrade if this was used extensively. For instance if analyzers where specified on a node level this might cause the index to be closed when imported (see #17187). In such a case all indices relying on this must be updated via `PUT /${index}/_settings`. Yet, this API has slightly different semantics since it overrides existing settings. To make this less painful this change adds a `preserve_existing` parameter on that API to ensure we have the same semantics as if the setting was applied on the node level. This change also adds a better error message and a change to the migration guide to ensure upgrades are smooth if index settings are specified on the node level. If a index setting is detected this change fails the node startup and prints a message like this: ``` ************************************************************************************* Found index level settings on node level configuration. Since elasticsearch 5.x index level settings can NOT be set on the nodes configuration like the elasticsearch.yaml, in system properties or command line arguments.In order to upgrade all indices the settings must be updated via the /${index}/_settings API. Unless all settings are dynamic all indices must be closed in order to apply the upgradeIndices created in the future should use index templates to set default values. Please ensure all required values are updated on all indices by executing: curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{ "index.number_of_shards" : "1", "index.query.default_field" : "main_field", "index.translog.durability" : "async", "index.ttl.disable_purge" : "true" }' ************************************************************************************* ```
2016-03-21 08:02:15 -04:00
==== Index Level Settings
In previous versions Elasticsearch allowed to specify index level setting
as _defaults_ on the node level, inside the `elasticsearch.yaml` file or even via
command-line parameters. From Elasticsearch 5.0 on only selected settings like
for instance `index.codec` can be set on the node level. All other settings must be
set on each individual index. To set default values on every index, index templates
should be used instead.
Reworked 5.0 breaking changes docs
2016-03-13 16:17:48 -04:00
==== Node settings
The `name` setting has been removed and is replaced by `node.name`. Usage of
`-Dname=some_node_name` is not supported anymore.
Remove ability to specify arbitrary node attributes with `node.` prefix Today the basic node settings like `node.data` and `node.master` can't really be fully validated since we allow to specify custom user attributes on the node level. We have to, in order to support that, add a wildcard setting for `node.*` to let these setting pass validation. Instead we should require a more contraint prefix like `node.attr.` that defines a namespace that is reserved for user attributes. This commit adds a new namespace for attributes in `node.attr`. Closes #17280
2016-03-30 05:43:04 -04:00
==== Node attribute settings
Node level attributes used for allocation filtering, forced awareness or other node identification / grouping
must be prefixed with `node.attr`. In previous versions it was possible to specify node attributes with the `node.`
prefix. All node attributes except of `node.master`, `node.data` and `node.ingest` must be moved to the new `node.attr.`
namespace.
add node.client breaking changes to migrate guide
2016-03-29 14:33:59 -04:00
==== Node types settings
The `node.client` setting has been removed. A node with such a setting set will not
start up. Instead, each node role needs to be set separately using the existing
`node.master`, `node.data` and `node.ingest` supported static settings.
update migration doc for removing gateway.format setting
2016-03-16 12:54:52 -04:00
==== Gateway settings
The `gateway.format` setting for configuring global and index state serialization
format has been removed. By default, `smile` is used as the format.
Reworked 5.0 breaking changes docs
2016-03-13 16:17:48 -04:00
==== Transport Settings
All settings with a `netty` infix have been replaced by their already existing
`transport` synonyms. For instance `transport.netty.bind_host` is no longer
supported and should be replaced by the superseding setting
`transport.bind_host`.
==== Script mode settings
Previously script mode settings (e.g., "script.inline: true",
"script.engine.groovy.inline.aggs: false", etc.) accepted the values
`on`, `true`, `1`, and `yes` for enabling a scripting mode, and the
values `off`, `false`, `0`, and `no` for disabling a scripting mode.
The variants `on`, `1`, and `yes ` for enabling and `off`, `0`,
and `no` for disabling are no longer supported.
==== Security manager settings
The option to disable the security manager `security.manager.enabled` has been
removed. In order to grant special permissions to elasticsearch users must
edit the local Java Security Policy.
==== Network settings
The `_non_loopback_` value for settings like `network.host` would arbitrarily
pick the first interface not marked as loopback. Instead, specify by address
scope (e.g. `_local_,_site_` for all loopback and private network addresses)
or by explicit interface names, hostnames, or addresses.
==== Forbid changing of thread pool types
Previously, <<modules-threadpool,thread pool types>> could be dynamically
adjusted. The thread pool type effectively controls the backing queue for the
thread pool and modifying this is an expert setting with minimal practical
benefits and high risk of being misused. The ability to change the thread pool
type for any thread pool has been removed. It is still possible to adjust
relevant thread pool parameters for each of the thread pools (e.g., depending
on the thread pool type, `keep_alive`, `queue_size`, etc.).
Remove suggest threadpool In #17198, we removed suggest transport action, which used the `suggest` threadpool to execute requests. Now `suggest` threadpool is unused and suggest requests are executed on the `search` threadpool.
2016-03-23 17:53:49 -04:00
==== Threadpool settings
The `suggest` threadpool has been removed, now suggest requests use the
`search` threadpool.
Reworked 5.0 breaking changes docs
2016-03-13 16:17:48 -04:00
==== Analysis settings
The `index.analysis.analyzer.default_index` analyzer is not supported anymore.
If you wish to change the analyzer to use for indexing, change the
`index.analysis.analyzer.default` analyzer instead.
Include pings from client nodes in master election We currently have a `discovery.zen.master_election.filter_client` setting that control whether their ping responses are ignored for master election (which is the current default). With the push to treat client nodes as normal nodes (and promote the transport/rest clients for client work), this should be changed. This commit remove this setting and it's companion `discovery.zen.master_election.filter_data` setting (currently defaulting to false) in favor of singe `discovery.zen.master_election.ignore_non_master_pings` setting with more intuitive name (defaulting to false). Resolves #17325 Closes #17329
2016-03-24 12:10:45 -04:00
==== Ping settings
Reworked 5.0 breaking changes docs
2016-03-13 16:17:48 -04:00
Previously, there were three settings for the ping timeout:
`discovery.zen.initial_ping_timeout`, `discovery.zen.ping.timeout` and
`discovery.zen.ping_timeout`. The former two have been removed and the only
setting key for the ping timeout is now `discovery.zen.ping_timeout`. The
default value for ping timeouts remains at three seconds.
Include pings from client nodes in master election We currently have a `discovery.zen.master_election.filter_client` setting that control whether their ping responses are ignored for master election (which is the current default). With the push to treat client nodes as normal nodes (and promote the transport/rest clients for client work), this should be changed. This commit remove this setting and it's companion `discovery.zen.master_election.filter_data` setting (currently defaulting to false) in favor of singe `discovery.zen.master_election.ignore_non_master_pings` setting with more intuitive name (defaulting to false). Resolves #17325 Closes #17329
2016-03-24 12:10:45 -04:00
`discovery.zen.master_election.filter_client` and `discovery.zen.master_election.filter_data` have
been removed in favor of the new `discovery.zen.master_election.ignore_non_master_pings`. This setting control how ping responses
are interpreted during master election and should be used with care and only in extreme cases. See documentation for details.
Reworked 5.0 breaking changes docs
2016-03-13 16:17:48 -04:00
==== Recovery settings
Recovery settings deprecated in 1.x have been removed:
* `index.shard.recovery.translog_size` is superseded by `indices.recovery.translog_size`
* `index.shard.recovery.translog_ops` is superseded by `indices.recovery.translog_ops`
* `index.shard.recovery.file_chunk_size` is superseded by `indices.recovery.file_chunk_size`
* `index.shard.recovery.concurrent_streams` is superseded by `indices.recovery.concurrent_streams`
* `index.shard.recovery.concurrent_small_file_streams` is superseded by `indices.recovery.concurrent_small_file_streams`
* `indices.recovery.max_size_per_sec` is superseded by `indices.recovery.max_bytes_per_sec`
If you are using any of these settings please take the time to review their
purpose. All of the settings above are considered _expert settings_ and should
only be used if absolutely necessary. If you have set any of the above setting
as persistent cluster settings please use the settings update API and set
their superseded keys accordingly.
The following settings have been removed without replacement
* `indices.recovery.concurrent_small_file_streams` - recoveries are now single threaded. The number of concurrent outgoing recoveries are throttled via allocation deciders
* `indices.recovery.concurrent_file_streams` - recoveries are now single threaded. The number of concurrent outgoing recoveries are throttled via allocation deciders
==== Translog settings
The `index.translog.flush_threshold_ops` setting is not supported anymore. In
order to control flushes based on the transaction log growth use
`index.translog.flush_threshold_size` instead.
Changing the translog type with `index.translog.fs.type` is not supported
anymore, the `buffered` implementation is now the only available option and
uses a fixed `8kb` buffer.
The translog by default is fsynced after every `index`, `create`, `update`,
`delete`, or `bulk` request. The ability to fsync on every operation is not
necessary anymore. In fact, it can be a performance bottleneck and it's trappy
since it enabled by a special value set on `index.translog.sync_interval`.
Now, `index.translog.sync_interval` doesn't accept a value less than `100ms`
which prevents fsyncing too often if async durability is enabled. The special
value `0` is no longer supported.
==== Request Cache Settings
The deprecated settings `index.cache.query.enable` and
`indices.cache.query.size` have been removed and are replaced with
`index.requests.cache.enable` and `indices.requests.cache.size` respectively.
`indices.requests.cache.clean_interval has been replaced with
`indices.cache.clean_interval and is no longer supported.
==== Field Data Cache Settings
The `indices.fielddata.cache.clean_interval` setting has been replaced with
`indices.cache.clean_interval`.
==== Allocation settings
The `cluster.routing.allocation.concurrent_recoveries` setting has been
replaced with `cluster.routing.allocation.node_concurrent_recoveries`.
==== Similarity settings
The 'default' similarity has been renamed to 'classic'.
==== Indexing settings
The `indices.memory.min_shard_index_buffer_size` and
`indices.memory.max_shard_index_buffer_size` have been removed as
Elasticsearch now allows any one shard to use amount of heap as long as the
total indexing buffer heap used across all shards is below the node's
`indices.memory.index_buffer_size` (defaults to 10% of the JVM heap).
==== Removed es.max-open-files
Setting the system property es.max-open-files to true to get
Elasticsearch to print the number of maximum open files for the
Elasticsearch process has been removed. This same information can be
obtained from the <<cluster-nodes-info>> API, and a warning is logged
on startup if it is set too low.
==== Removed es.netty.gathering
Fix typo inadvertently introduced
2016-03-15 10:05:28 -04:00
Disabling Netty from using NIO gathering could be done via the escape
Reworked 5.0 breaking changes docs
2016-03-13 16:17:48 -04:00
hatch of setting the system property "es.netty.gathering" to "false".
Time has proven enabling gathering by default is a non-issue and this
non-documented setting has been removed.
==== Removed es.useLinkedTransferQueue
The system property `es.useLinkedTransferQueue` could be used to
control the queue implementation used in the cluster service and the
handling of ping responses during discovery. This was an undocumented
setting and has been removed.
==== Cache concurrency level settings removed
Two cache concurrency level settings
`indices.requests.cache.concurrency_level` and
`indices.fielddata.cache.concurrency_level` because they no longer apply to
the cache implementation used for the request cache and the field data cache.
Bootstrap does not set system properties Today, certain bootstrap properties are set and read via system properties. This action-at-distance way of managing these properties is rather confusing, and completely unnecessary. But another problem exists with setting these as system properties. Namely, these system properties are interpreted as Elasticsearch settings, not all of which are registered. This leads to Elasticsearch failing to startup if any of these special properties are set. Instead, these properties should be kept as local as possible, and passed around as method parameters where needed. This eliminates the action-at-distance way of handling these properties, and eliminates the need to register these non-setting properties. This commit does exactly that. Additionally, today we use the "-D" command line flag to set the properties, but this is confusing because "-D" is a special flag to the JVM for setting system properties. This creates confusion because some "-D" properties should be passed via arguments to the JVM (so via ES_JAVA_OPTS), and some should be passed as arguments to Elasticsearch. This commit changes the "-D" flag for Elasticsearch settings to "-E".
2016-03-13 09:10:56 -04:00
==== Using system properties to configure Elasticsearch
Elasticsearch can be configured by setting system properties on the
command line via `-Des.name.of.property=value.of.property`. This will be
removed in a future version of Elasticsearch. Instead, use
`-E es.name.of.setting=value.of.setting`. Note that in all cases the
name of the setting must be prefixed with `es.`.
==== Removed using double-dashes to configure Elasticsearch
Elasticsearch could previously be configured on the command line by
setting settings via `--name.of.setting value.of.setting`. This feature
has been removed. Instead, use
`-Ees.name.of.setting=value.of.setting`. Note that in all cases the
name of the setting must be prefixed with `es.`.
Enforce `discovery.zen.minimum_master_nodes` is set when bound to a public ip #17288 discovery.zen.minimum_master_nodes is the single most important setting to set on a production cluster. We have no way of supplying a good default so it must be set by the user. Binding a node to a public IP (as opposed to the default local host) is a good enough indication that a node will be part of a production cluster cluster and thus it's a good tradeoff to enforce the settings. Note that nothing prevent users from setting it to 1 in a single node cluster. Closes #17288
2016-03-23 11:43:30 -04:00
==== Discovery Settings
[docs] clarify where discovery.zen.minimum_master_node is required https://github.com/elastic/elasticsearch/pull/17288 added a check to enforce that the `discovery.zen.minimum_master_nodes` configuration is set when nodes have the `host`, `port`, or `bind_host` set in either `transport` or general `network` configuration sections. This was documented incorrectly as "nodes that are bound to a non-loopback interface", which lead to confusion as I set `network.host: "localhost"` and the check was still failing. This change updates the docs to detail the actual check. I think it also highlights how complex the check is and the need for a simpler solution.
2016-03-28 15:44:42 -04:00
The `discovery.zen.minimum_master_node` must bet set for nodes that have
`network.host`, `network.bind_host`, `network.publish_host`,
`transport.host`, `transport.bind_host`, or `transport.publish_host`
configuration options set. We see those nodes as in "production" mode
and thus require the setting.
settings: Removed `action.get.realtime` setting Closes #12543
2016-04-19 09:06:14 -04:00
==== Realtime get setting
The `action.get.realtime` setting has been removed. This setting was
a fallback realtime setting for the get and mget APIs when realtime
wasn't specified. Now if the parameter isn't specified we always
default to true.