honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Document the index corruption bug that gets fixed via Lucene 8.5.1. (#55232) 

Using soft deletes on shrunk indices may cause corruption.
This commit is contained in:
Adrien Grand 2020-04-17 13:37:37 +02:00 committed by GitHub
parent e77256c50f
commit 0cb6a1f089
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 74 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/reference/release-notes/7.0.asciidoc
View File

@ -9,6 +9,14 @@ Also see <<breaking-changes-7.0,Breaking changes in 7.0>>.
[float] [float]
=== Known issues === Known issues
* Applying deletes or updates on an index after it has been shrunk may corrupt
the index. In order to prevent this issue, it is recommended to stop shrinking
read-write indices. For read-only indices, it is recommended to force-merge
indices after shrinking, which significantly reduces the likeliness of this
corruption in the case that deletes/updates would be applied by mistake. This
bug is fixed in {es} 7.7 and later versions. More details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats * Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
https://github.com/elastic/elasticsearch/pull/52555 https://github.com/elastic/elasticsearch/pull/52555

8
docs/reference/release-notes/7.1.asciidoc
View File

@ -6,6 +6,14 @@ Also see <<breaking-changes-7.1,Breaking changes in 7.1>>.
[float] [float]
=== Known issues === Known issues
* Applying deletes or updates on an index after it has been shrunk may corrupt
the index. In order to prevent this issue, it is recommended to stop shrinking
read-write indices. For read-only indices, it is recommended to force-merge
indices after shrinking, which significantly reduces the likeliness of this
corruption in the case that deletes/updates would be applied by mistake. This
bug is fixed in {es} 7.7 and later versions. More details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats * Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
https://github.com/elastic/elasticsearch/pull/52555 https://github.com/elastic/elasticsearch/pull/52555

8
docs/reference/release-notes/7.2.asciidoc
View File

@ -88,6 +88,14 @@ Also see <<breaking-changes-7.2,Breaking changes in 7.2>>.
[float] [float]
=== Known issues === Known issues
* Applying deletes or updates on an index after it has been shrunk may corrupt
the index. In order to prevent this issue, it is recommended to stop shrinking
read-write indices. For read-only indices, it is recommended to force-merge
indices after shrinking, which significantly reduces the likeliness of this
corruption in the case that deletes/updates would be applied by mistake. This
bug is fixed in {es} 7.7 and later versions. More details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats * Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
https://github.com/elastic/elasticsearch/pull/52555 https://github.com/elastic/elasticsearch/pull/52555

8
docs/reference/release-notes/7.3.asciidoc
View File

@ -143,6 +143,14 @@ Also see <<breaking-changes-7.3,Breaking changes in 7.3>>.
[float] [float]
=== Known issues === Known issues
* Applying deletes or updates on an index after it has been shrunk may corrupt
the index. In order to prevent this issue, it is recommended to stop shrinking
read-write indices. For read-only indices, it is recommended to force-merge
indices after shrinking, which significantly reduces the likeliness of this
corruption in the case that deletes/updates would be applied by mistake. This
bug is fixed in {es} 7.7 and later versions. More details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats * Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
https://github.com/elastic/elasticsearch/pull/52555 https://github.com/elastic/elasticsearch/pull/52555

8
docs/reference/release-notes/7.4.asciidoc
View File

@ -111,6 +111,14 @@ Also see <<breaking-changes-7.4,Breaking changes in 7.4>>.
[float] [float]
=== Known issues === Known issues
* Applying deletes or updates on an index after it has been shrunk may corrupt
the index. In order to prevent this issue, it is recommended to stop shrinking
read-write indices. For read-only indices, it is recommended to force-merge
indices after shrinking, which significantly reduces the likeliness of this
corruption in the case that deletes/updates would be applied by mistake. This
bug is fixed in {es} 7.7 and later versions. More details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
* Activating the <<search-slow-log, search slow log>> should be avoided in this version. * Activating the <<search-slow-log, search slow log>> should be avoided in this version.
Any attempt to log a slow search can throw an AIOOBE due to a bug that Any attempt to log a slow search can throw an AIOOBE due to a bug that
performs concurrent modifications on a shared byte array. performs concurrent modifications on a shared byte array.

8
docs/reference/release-notes/7.5.asciidoc
View File

@ -94,6 +94,14 @@ Also see <<breaking-changes-7.5,Breaking changes in 7.5>>.
[float] [float]
=== Known issues === Known issues
* Applying deletes or updates on an index after it has been shrunk may corrupt
the index. In order to prevent this issue, it is recommended to stop shrinking
read-write indices. For read-only indices, it is recommended to force-merge
indices after shrinking, which significantly reduces the likeliness of this
corruption in the case that deletes/updates would be applied by mistake. This
bug is fixed in {es} 7.7 and later versions. More details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
* Stop all {transforms} during a rolling upgrade to 7.5. * Stop all {transforms} during a rolling upgrade to 7.5.
If a {transform} is running during upgrade, the {transform} audit index might disappear. If a {transform} is running during upgrade, the {transform} audit index might disappear.
(issue: {issue}/49730[#49730]) (issue: {issue}/49730[#49730])

8
docs/reference/release-notes/7.6.asciidoc
View File

@ -155,6 +155,14 @@ Also see <<breaking-changes-7.6,Breaking changes in 7.6>>.
[float] [float]
=== Known issues === Known issues
* Applying deletes or updates on an index after it has been shrunk may corrupt
the index. In order to prevent this issue, it is recommended to stop shrinking
read-write indices. For read-only indices, it is recommended to force-merge
indices after shrinking, which significantly reduces the likeliness of this
corruption in the case that deletes/updates would be applied by mistake. This
bug is fixed in {es} 7.7 and later versions. More details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats * Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
https://github.com/elastic/elasticsearch/pull/52555 https://github.com/elastic/elasticsearch/pull/52555

1
docs/reference/release-notes/7.7.asciidoc
View File

@ -447,6 +447,7 @@ Engine::
* Fix doc_stats and segment_stats of ReadOnlyEngine {pull}53345[#53345] (issues: {issue}51303[#51303], {issue}51331[#51331]) * Fix doc_stats and segment_stats of ReadOnlyEngine {pull}53345[#53345] (issues: {issue}51303[#51303], {issue}51331[#51331])
* Do not wrap soft-deletes reader for segment stats {pull}51331[#51331] (issues: {issue}51192[#51192], {issue}51303[#51303]) * Do not wrap soft-deletes reader for segment stats {pull}51331[#51331] (issues: {issue}51192[#51192], {issue}51303[#51303])
* Account soft-deletes in FrozenEngine {pull}51192[#51192] (issue: {issue}50775[#50775]) * Account soft-deletes in FrozenEngine {pull}51192[#51192] (issue: {issue}50775[#50775])
* Fixed an index corruption bug that would occur when applying deletes or updates on an index after it has been shrunk. More details can be found on the https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
Features/CAT APIs:: Features/CAT APIs::
* Fix NPE in RestPluginsAction {pull}52620[#52620] (issue: {issue}45321[#45321]) * Fix NPE in RestPluginsAction {pull}52620[#52620] (issue: {issue}45321[#45321])

17
docs/reference/release-notes/highlights-7.7.0.asciidoc
View File

@ -7,6 +7,23 @@
//NOTE: The notable-highlights tagged regions are re-used in the //NOTE: The notable-highlights tagged regions are re-used in the
//Installation and Upgrade Guide //Installation and Upgrade Guide
// tag::notable-highlights[]
[float]
=== Fixed index corruption on shrunk indices
Applying deletes or updates on an index after it had been shrunk would likely
corrupt the index. We advise users of Elasticsearch 6.x who opt in for soft
deletes on some of their indices and all users of Elasticsearch 7.x to upgrade
to 7.7 as soon as possible to no longer be subject to this corruption bug. In
case upgrading in the near future is not an option, we recommend to completely
stop using `_shrink` on read-write indices and to do a force-merge right after
shrinking on read-only indices, which significantly reduces the likeliness of
being affected by this bug in case deletes or updates get applied by mistake.
This bug is fixed as of {es} 7.7.0. Low-level details can be found on the
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
// end::notable-highlights[]
// tag::notable-highlights[] // tag::notable-highlights[]
[discrete] [discrete]
=== {transforms-cap} === {transforms-cap}