mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-06 04:58:50 +00:00
033ba725af
Elasticsearch has long [supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-versioning) compare and set (a.k.a optimistic concurrency control) operations using internal document versioning. Sadly that approach is flawed and can sometime do the wrong thing. Here's the relevant excerpt from the resiliency status page: > When a primary has been partitioned away from the cluster there is a short period of time until it detects this. During that time it will continue indexing writes locally, thereby updating document versions. When it tries to replicate the operation, however, it will discover that it is partitioned away. It won’t acknowledge the write and will wait until the partition is resolved to negotiate with the master on how to proceed. The master will decide to either fail any replicas which failed to index the operations on the primary or tell the primary that it has to step down because a new primary has been chosen in the meantime. Since the old primary has already written documents, clients may already have read from the old primary before it shuts itself down. The version numbers of these reads may not be unique if the new primary has already accepted writes for the same document We recently [introduced](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/optimistic-concurrency-control.html) a new sequence number based approach that doesn't suffer from this dirty reads problem. This commit removes support for internal versioning as a concurrency control mechanism in favor of the sequence number approach. Relates to #1078
157 lines
7.0 KiB
Plaintext
157 lines
7.0 KiB
Plaintext
[float]
|
|
[[breaking_70_api_changes]]
|
|
=== API changes
|
|
|
|
[float]
|
|
==== Internal Versioning is no longer supported for optimistic concurrency control
|
|
|
|
Elasticsearch maintains a numeric version field for each document it stores. That field
|
|
is incremented by one with every change to the document. Until 7.0.0 the API allowed using
|
|
that field for optimistic concurrency control, i.e., making a write operation conditional
|
|
on the current document version. Sadly, that approach is flawed because the value of the
|
|
version doesn't always uniquely represent a change to the document. If a primary fails
|
|
while handling a write operation, it may expose a version that will then be reused by the
|
|
new primary.
|
|
|
|
Due to that issue, internal versioning can no longer be used and is replaced by a new
|
|
method based on sequence numbers. See <<optimistic-concurrency-control>> for more details.
|
|
|
|
Note that the `external` versioning type is still fully supported.
|
|
|
|
[float]
|
|
==== Camel case and underscore parameters deprecated in 6.x have been removed
|
|
A number of duplicate parameters deprecated in 6.x have been removed from
|
|
Bulk request, Multi Get request, Term Vectors request, and More Like This Query
|
|
requests.
|
|
|
|
The following camel case parameters have been removed:
|
|
|
|
* `opType`
|
|
* `versionType`, `_versionType`
|
|
|
|
The following parameters starting with underscore have been removed:
|
|
|
|
* `_parent`
|
|
* `_retry_on_conflict`
|
|
* `_routing`
|
|
* `_version`
|
|
* `_version_type`
|
|
|
|
Instead of these removed parameters, use their non camel case equivalents without
|
|
starting underscore, e.g. use `version_type` instead of `_version_type` or `versionType`.
|
|
|
|
[float]
|
|
==== Thread pool info
|
|
|
|
In previous versions of Elasticsearch, the thread pool info returned in the
|
|
<<cluster-nodes-info,nodes info API>> returned `min` and `max` values reflecting
|
|
the configured minimum and maximum number of threads that could be in each
|
|
thread pool. The trouble with this representation is that it does not align with
|
|
the configuration parameters used to configure thread pools. For
|
|
<<modules-threadpool,scaling thread pools>>, the minimum number of threads is
|
|
configured by a parameter called `core` and the maximum number of threads is
|
|
configured by a parameter called `max`. For <<modules-threadpool,fixed thread
|
|
pools>>, there is only one configuration parameter along these lines and that
|
|
parameter is called `size`, reflecting the fixed number of threads in the
|
|
pool. This discrepancy between the API and the configuration parameters has been
|
|
rectified. Now, the API will report `core` and `max` for scaling thread pools,
|
|
and `size` for fixed thread pools.
|
|
|
|
Similarly, in the cat thread pool API the existing `size` output has been
|
|
renamed to `pool_size` which reflects the number of threads currently in the
|
|
pool; the shortcut for this value has been changed from `s` to `psz`. The `min`
|
|
output has been renamed to `core` with a shortcut of `cr`, the shortcut for
|
|
`max` has been changed to `mx`, and the `size` output with a shortcut of `sz`
|
|
has been reused to report the configured number of threads in the pool. This
|
|
aligns the output of the API with the configuration values for thread
|
|
pools. Note that `core` and `max` will be populated for scaling thread pools,
|
|
and `size` will be populated for fixed thread pools.
|
|
|
|
[float]
|
|
==== The parameter `fields` deprecated in 6.x has been removed from Bulk request
|
|
and Update request. The Update API returns `400 - Bad request` if request contains
|
|
unknown parameters (instead of ignored in the previous version).
|
|
|
|
[float]
|
|
[[remove-suggest-metric]]
|
|
==== Remove support for `suggest` metric/index metric in indices stats and nodes stats APIs
|
|
|
|
Previously, `suggest` stats were folded into `search` stats. Support for the
|
|
`suggest` metric on the indices stats and nodes stats APIs remained for
|
|
backwards compatibility. Backwards support for the `suggest` metric was
|
|
deprecated in 6.3.0 and now removed in 7.0.0.
|
|
|
|
[[remove-field-caps-body]]
|
|
|
|
In the past, `fields` could be provided either as a parameter, or as part of the request
|
|
body. Specifying `fields` in the request body as opposed to a parameter was deprecated
|
|
in 6.4.0, and is now unsupported in 7.0.0.
|
|
|
|
[float]
|
|
==== `copy_settings` is deprecated on shrink and split APIs
|
|
|
|
Versions of Elasticsearch prior to 6.4.0 did not copy index settings on shrink
|
|
and split operations. Starting with Elasticsearch 7.0.0, the default behavior
|
|
will be for such settings to be copied on such operations. To enable users in
|
|
6.4.0 to transition in 6.4.0 to the default behavior in 7.0.0, the
|
|
`copy_settings` parameter was added on the REST layer. As this behavior will be
|
|
the only behavior in 8.0.0, this parameter is deprecated in 7.0.0 for removal in
|
|
8.0.0.
|
|
|
|
[float]
|
|
==== The deprecated stored script contexts have now been removed
|
|
When putting stored scripts, support for storing them with the deprecated `template` context or without a context is
|
|
now removed. Scripts must be stored using the `script` context as mentioned in the documentation.
|
|
|
|
[float]
|
|
==== Removed Get Aliases API limitations when {security-features} are enabled
|
|
|
|
The behavior and response codes of the get aliases API no longer vary
|
|
depending on whether {security-features} are enabled. Previously a
|
|
404 - NOT FOUND (IndexNotFoundException) could be returned in case the
|
|
current user was not authorized for any alias. An empty response with
|
|
status 200 - OK is now returned instead at all times.
|
|
|
|
[float]
|
|
==== Put User API response no longer has `user` object
|
|
|
|
The Put User API response was changed in 6.5.0 to add the `created` field
|
|
outside of the user object where it previously had been. In 7.0.0 the user
|
|
object has been removed in favor of the top level `created` field.
|
|
|
|
[float]
|
|
==== Source filtering url parameters `_source_include` and `_source_exclude` have been removed
|
|
|
|
The deprecated in 6.x url parameters are now removed. Use `_source_includes` and `_source_excludes` instead.
|
|
|
|
[float]
|
|
==== Multi Search Request metadata validation
|
|
|
|
MultiSearchRequests issued through `_msearch` now validate all keys in the metadata section. Previously unknown keys were ignored
|
|
while now an exception is thrown.
|
|
|
|
[float]
|
|
==== Deprecated graph endpoints removed
|
|
|
|
The deprecated graph endpoints (those with `/_graph/_explore`) have been
|
|
removed.
|
|
|
|
|
|
[float]
|
|
==== Deprecated `_termvector` endpoint removed
|
|
|
|
The `_termvector` endpoint was deprecated in 2.0 and has now been removed.
|
|
The endpoint `_termvectors` (plural) should be used instead.
|
|
|
|
[float]
|
|
==== When {security-features} are enabled, index monitoring APIs over restricted indices are not authorized implicitly anymore
|
|
|
|
Restricted indices (currently only `.security-6` and `.security`) are special internal
|
|
indices that require setting the `allow_restricted_indices` flag on every index
|
|
permission that covers them. If this flag is `false` (default) the permission
|
|
will not cover these and actions against them will not be authorized.
|
|
However, the monitoring APIs were the only exception to this rule. This exception
|
|
has been forfeited and index monitoring privileges have to be granted explicitly,
|
|
using the `allow_restricted_indices` flag on the permission (as any other index
|
|
privilege).
|