honeymoose/OpenSearch
1
0
Fork 0
mirror of https://github.com/honeymoose/OpenSearch.git synced 2025-03-25 09:28:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added upgrade docs explaining how to reindex in place or reindex from remote

Browse Source 
Closes #20675
This commit is contained in:
Clinton Gormley 2016-10-11 12:14:35 +02:00
parent e4c7d8183e
commit 02a739d3c9
5 changed files with 179 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/reference/docs/reindex.asciidoc
View File

@ -367,6 +367,7 @@ POST _reindex
// TEST[s/^/PUT source\n/]
[float]
[[reindex-from-remote]]
=== Reindex from Remote
Reindex supports reindexing from a remote Elasticsearch cluster:

35
docs/reference/setup/cluster_restart.asciidoc
View File

@ -8,7 +8,9 @@ required.
The process to perform an upgrade with a full cluster restart is as follows:
==== Step 1: Disable shard allocation
. *Disable shard allocation*
+
--
When you shut down a node, the allocation process will immediately try to
replicate the shards that were on that node to other nodes in the cluster,
@ -26,8 +28,11 @@ PUT _cluster/settings
--------------------------------------------------
// CONSOLE
// TEST[skip:indexes don't assign]
--
==== Step 2: Perform a synced flush
. *Perform a synced flush*
+
--
Shard recovery will be much faster if you stop indexing and issue a
<<indices-synced-flush, synced-flush>> request:
@ -41,19 +46,28 @@ POST _flush/synced
A synced flush request is a ``best effort'' operation. It will fail if there
are any pending indexing operations, but it is safe to reissue the request
multiple times if necessary.
--
==== Step 3: Shutdown and upgrade all nodes
. *Shutdown and upgrade all nodes*
+
--
Stop all Elasticsearch services on all nodes in the cluster. Each node can be
upgraded following the same procedure described in <<upgrade-node>>.
--
==== Step 4: Upgrade any plugins
. *Upgrade any plugins*
+
--
Elasticsearch plugins must be upgraded when upgrading a node. Use the
`elasticsearch-plugin` script to install the correct version of any plugins
that you need.
--
==== Step 5: Start the cluster
. *Start the cluster*
+
--
If you have dedicated master nodes -- nodes with `node.master` set to
`true`(the default) and `node.data` set to `false` -- then it is a good idea
@ -75,8 +89,11 @@ GET _cat/nodes
// CONSOLE
Use these APIs to check that all nodes have successfully joined the cluster.
--
==== Step 6: Wait for yellow
. *Wait for yellow*
+
--
As soon as each node has joined the cluster, it will start to recover any
primary shards that are stored locally. Initially, the
@ -87,8 +104,11 @@ Once each node has recovered its local shards, the `status` will become
`yellow`, meaning all primary shards have been recovered, but not all replica
shards are allocated. This is to be expected because allocation is still
disabled.
--
==== Step 7: Reenable allocation
. *Reenable allocation*
+
--
Delaying the allocation of replicas until all nodes have joined the cluster
allows the master to allocate replicas to nodes which already have local shard
@ -124,3 +144,4 @@ GET _cat/recovery
Once the `status` column in the `_cat/health` output has reached `green`, all
primary and replica shards have been successfully allocated.
--

92
docs/reference/setup/reindex_upgrade.asciidoc Normal file
View File

@ -0,0 +1,92 @@
[[reindex-upgrade]]
=== Reindex to upgrade
Elasticsearch is able to use indices created in the previous major version
only. For instance, Elasticsearch 6.x can use indices created in
Elasticsearch 5.x, but not those created in Elasticsearch 2.x or before.
NOTE: Elasticsearch 6.x nodes will fail to start in the presence of too old indices.
If you are running an Elasticsearch 5.x cluster which contains indices that
were created before 5.x, you will either need to delete those old indices or
to reindex them before upgrading to 6.x. See <<reindex-upgrade-inplace>>.
If you are running an Elasticsearch 2.x cluster or older, you have two options:
* First upgrade to Elasticsearch 5.x, reindex the old indices, then upgrade
to 6.x. See <<reindex-upgrade-inplace>>.
* Create a new 6.x cluster and use reindex-from-remote to import indices
directly from the 2.x cluster. See <<reindex-upgrade-remote>>.
[[reindex-upgrade-inplace]]
==== Reindex in place
If you are running a 5.x cluster which contains indices created in
Elasticsearch 2.x, you will need to reindex (or delete) those indices before
upgrading to Elasticsearch 6.x.
The reindex process works as follows:
* Create a new index, copying the mappings and settings from the old index.
Set the `refresh_interval` to `-1` and the `number_of_replicas` to `0` for
efficient reindexing.
* Reindex all documents from the old index to the new index using the
<<docs-reindex,reindex API>>.
* Reset the `refresh_interval` and `number_of_replicas` to the values
used in the old index, and wait for the index to become green.
* In a single <<indices-aliases,update aliases>> request:
* Delete the old index.
* Add an alias with the old index name to the new index.
* Add any aliases that existed on the old index to the new index.
At the end of this process, you will have a new 5.x index which can be used
by an Elasticsearch 6.x cluster.
[[reindex-upgrade-remote]]
==== Upgrading with reindex-from-remote
If you are running a 1.x or 2.x cluster and would like to migrate directly to 6.x
without first migrating to 5.x, you can do so using
<<reindex-from-remote,reindex-from-remote>>.
[WARNING]
=============================================
Elasticsearch includes backwards compatibility code that allows indices from
the previous major version to be upgraded to the current major version. By
moving directly from Elasticsearch 2.x or before to 6.x, you will have to solve any
backwards compatibility issues yourself.
=============================================
You will need to set up a 6.x cluster alongside your existing old cluster.
The 6.x cluster needs to have access to the REST API of the old cluster.
For each old index that you want to transfer to the 6.x cluster, you will need
to:
* Create a new index in 6.x with the appropriate mappings and settings. Set
the `refresh_interval` to `-1` and set `number_of_replicas` to `0` for
faster reindexing.
* Use <<reindex-from-remote,reindex-from-remote>> to pull documents from the
old index into the new 6.x index.
* If you run the reindex job in the background (with `wait_for_completion` set
to `false`), the reindex request will return a `task_id` which can be used to
monitor progress of the reindex job in the <<tasks,task API>>:
`GET _tasks/TASK_ID`.
* Once reindex has completed, set the `refresh_interval` and
`number_of_replicas` to the desired values (the defaults are `30s` and `1`
respectively).
* Once the new index has finished replication, you can delete the old index.
The 6.x cluster can start out small, and you can gradually move nodes from the
old cluster to the 6.x cluster as you migrate indices across.

41
docs/reference/setup/rolling_upgrade.asciidoc
View File

@ -12,7 +12,9 @@ supported for your version of Elasticsearch.
To perform a rolling upgrade:
==== Step 1: Disable shard allocation
. *Disable shard allocation*
+
--
When you shut down a node, the allocation process will wait for one minute
before starting to replicate the shards that were on that node to other nodes
@ -30,8 +32,11 @@ PUT _cluster/settings
--------------------------------------------------
// CONSOLE
// TEST[skip:indexes don't assign]
--
==== Step 2: Stop non-essential indexing and perform a synced flush (Optional)
. *Stop non-essential indexing and perform a synced flush (Optional)*
+
--
You may happily continue indexing during the upgrade. However, shard recovery
will be much faster if you temporarily stop non-essential indexing and issue a
@ -46,9 +51,11 @@ POST _flush/synced
A synced flush request is a ``best effort'' operation. It will fail if there
are any pending indexing operations, but it is safe to reissue the request
multiple times if necessary.
--
[[upgrade-node]]
==== Step 3: Stop and upgrade a single node
. [[upgrade-node]] *Stop and upgrade a single node*
+
--
Shut down one of the nodes in the cluster *before* starting the upgrade.
@ -87,14 +94,20 @@ To upgrade using a zip or compressed tarball:
* Either copy the files in the `data` directory from your old installation
to your new installation, or configure the location of the data directory
in the `config/elasticsearch.yml` file, with the `path.data` setting.
--
==== Step 4: Upgrade any plugins
. *Upgrade any plugins*
+
--
Elasticsearch plugins must be upgraded when upgrading a node. Use the
`elasticsearch-plugin` script to install the correct version of any plugins
that you need.
--
==== Step 5: Start the upgraded node
. *Start the upgraded node*
+
--
Start the now upgraded node and confirm that it joins the cluster by checking
the log file or by checking the output of this request:
@ -104,8 +117,11 @@ the log file or by checking the output of this request:
GET _cat/nodes
--------------------------------------------------
// CONSOLE
--
==== Step 6: Reenable shard allocation
. *Reenable shard allocation*
+
--
Once the node has joined the cluster, reenable shard allocation to start using
the node:
@ -120,8 +136,11 @@ PUT _cluster/settings
}
--------------------------------------------------
// CONSOLE
--
==== Step 7: Wait for the node to recover
. *Wait for the node to recover*
+
--
You should wait for the cluster to finish shard allocation before upgrading
the next node. You can check on progress with the <<cat-health,`_cat/health`>>
@ -168,8 +187,12 @@ GET _cat/recovery
If you stopped indexing, then it is safe to resume indexing as soon as
recovery has completed.
--
==== Step 8: Repeat
. *Repeat*
+
--
When the cluster is stable and the node has recovered, repeat the above steps
for all remaining nodes.
--

32
docs/reference/setup/upgrade.asciidoc
View File

@ -1,5 +1,5 @@
[[setup-upgrade]]
== Upgrading
== Upgrading Elasticsearch
[IMPORTANT]
===========================================
@ -22,14 +22,34 @@ consult this table:
[cols="1<m,1<m,3",options="header",]
|=======================================================================
|Upgrade From |Upgrade To |Supported Upgrade Type
|2.x |2.y |<<rolling-upgrades,Rolling upgrade>> (where `y > x`)
|< 5.x |6.x |<<reindex-upgrade,Reindex to upgrade>>
|5.x |5.y |<<rolling-upgrades,Rolling upgrade>> (where `y > x`)
|2.x |5.x |<<restart-upgrade,Full cluster restart>>
|5.0.0-alpha1 |5.y |<<restart-upgrade,Full cluster restart>>
|5.0.0-alpha2 |5.y |<<restart-upgrade,Full cluster restart>>
|5.0.0-beta1 |5.y |<<restart-upgrade,Full cluster restart>>
|5.x |6.x |<<restart-upgrade,Full cluster restart>>
|6.0.0 pre GA |6.x |<<restart-upgrade,Full cluster restart>>
|6.x |6.y |<<rolling-upgrades,Rolling upgrade>> (where `y > x`)
|=======================================================================
[IMPORTANT]
.Indices created in Elasticsearch 2.x or before
===============================================
Elasticsearch is able to read indices created in the *previous major version
only*. For instance, Elasticsearch 6.x can use indices created in
Elasticsearch 5.x, but not those created in Elasticsearch 2.x or before.
This condition also applies to indices backed up with
<<modules-snapshots,snapshot and restore>>. If an index was originally
created in 2.x, it cannot be restored into a 6.x cluster even if the
snapshot was made by a 5.x cluster.
Elasticsearch 6.x nodes will fail to start in the presence of too old indices.
See <<reindex-upgrade>> for more information about how to upgrade old indices.
===============================================
include::rolling_upgrade.asciidoc[]
include::cluster_restart.asciidoc[]
include::reindex_upgrade.asciidoc[]