189 lines
5.7 KiB
Plaintext
189 lines
5.7 KiB
Plaintext
[[rolling-upgrades]]
|
|
== Rolling upgrades
|
|
|
|
A rolling upgrade allows an {es} cluster to be upgraded one node at
|
|
a time so upgrading does not interrupt service. Running multiple versions of
|
|
{es} in the same cluster beyond the duration of an upgrade is
|
|
not supported, as shards cannot be replicated from upgraded nodes to nodes
|
|
running the older version.
|
|
|
|
Rolling upgrades are supported:
|
|
|
|
* Between minor versions
|
|
* https://www.elastic.co/guide/en/elastic-stack/6.8/upgrading-elastic-stack.html[From 5.6 to 6.8]
|
|
* From 6.8 to {version}
|
|
|
|
Upgrading directly to {version} from 6.7 or earlier requires a
|
|
<<restart-upgrade, full cluster restart>>.
|
|
|
|
To perform a rolling upgrade from 6.8 to {version}:
|
|
|
|
. *Disable shard allocation*.
|
|
+
|
|
--
|
|
include::disable-shard-alloc.asciidoc[]
|
|
--
|
|
|
|
. *Stop non-essential indexing and perform a synced flush.* (Optional)
|
|
+
|
|
--
|
|
While you can continue indexing during the upgrade, shard recovery
|
|
is much faster if you temporarily stop non-essential indexing and perform a
|
|
<<indices-synced-flush, synced-flush>>.
|
|
|
|
include::synced-flush.asciidoc[]
|
|
|
|
--
|
|
|
|
. *Temporarily stop the tasks associated with active {ml} jobs and {dfeeds}.* (Optional)
|
|
+
|
|
--
|
|
include::close-ml.asciidoc[]
|
|
--
|
|
|
|
. [[upgrade-node]] *Shut down a single node*.
|
|
+
|
|
--
|
|
include::shut-down-node.asciidoc[]
|
|
--
|
|
|
|
. *Upgrade the node you shut down.*
|
|
+
|
|
--
|
|
include::upgrade-node.asciidoc[]
|
|
include::set-paths-tip.asciidoc[]
|
|
--
|
|
|
|
. *Upgrade any plugins.*
|
|
+
|
|
Use the `elasticsearch-plugin` script to install the upgraded version of each
|
|
installed {es} plugin. All plugins must be upgraded when you upgrade
|
|
a node.
|
|
|
|
. If you use {es} {security-features} to define realms, verify that your realm
|
|
settings are up-to-date. The format of realm settings changed in version 7.0, in
|
|
particular, the placement of the realm type changed. See
|
|
<<realm-settings,Realm settings>>.
|
|
|
|
. *Start the upgraded node.*
|
|
+
|
|
--
|
|
|
|
Start the newly-upgraded node and confirm that it joins the cluster by checking
|
|
the log file or by submitting a `_cat/nodes` request:
|
|
|
|
[source,sh]
|
|
--------------------------------------------------
|
|
GET _cat/nodes
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
--
|
|
|
|
. *Reenable shard allocation.*
|
|
+
|
|
--
|
|
|
|
Once the node has joined the cluster, remove the `cluster.routing.allocation.enable`
|
|
setting to enable shard allocation and start using the node:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT _cluster/settings
|
|
{
|
|
"persistent": {
|
|
"cluster.routing.allocation.enable": null
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
--
|
|
|
|
. *Wait for the node to recover.*
|
|
+
|
|
--
|
|
|
|
Before upgrading the next node, wait for the cluster to finish shard allocation.
|
|
You can check progress by submitting a <<cat-health,`_cat/health`>> request:
|
|
|
|
[source,sh]
|
|
--------------------------------------------------
|
|
GET _cat/health?v
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
Wait for the `status` column to switch from `yellow` to `green`. Once the
|
|
node is `green`, all primary and replica shards have been allocated.
|
|
|
|
[IMPORTANT]
|
|
====================================================
|
|
During a rolling upgrade, primary shards assigned to a node running the new
|
|
version cannot have their replicas assigned to a node with the old
|
|
version. The new version might have a different data format that is
|
|
not understood by the old version.
|
|
|
|
If it is not possible to assign the replica shards to another node
|
|
(there is only one upgraded node in the cluster), the replica
|
|
shards remain unassigned and status stays `yellow`.
|
|
|
|
In this case, you can proceed once there are no initializing or relocating shards
|
|
(check the `init` and `relo` columns).
|
|
|
|
As soon as another node is upgraded, the replicas can be assigned and the
|
|
status will change to `green`.
|
|
====================================================
|
|
|
|
Shards that were not <<indices-synced-flush,sync-flushed>> might take longer to
|
|
recover. You can monitor the recovery status of individual shards by
|
|
submitting a <<cat-recovery,`_cat/recovery`>> request:
|
|
|
|
[source,sh]
|
|
--------------------------------------------------
|
|
GET _cat/recovery
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
If you stopped indexing, it is safe to resume indexing as soon as
|
|
recovery completes.
|
|
--
|
|
|
|
. *Repeat*
|
|
+
|
|
--
|
|
|
|
When the node has recovered and the cluster is stable, repeat these steps
|
|
for each node that needs to be updated.
|
|
|
|
--
|
|
|
|
. *Restart machine learning jobs.*
|
|
+
|
|
--
|
|
include::open-ml.asciidoc[]
|
|
--
|
|
|
|
|
|
[IMPORTANT]
|
|
====================================================
|
|
|
|
During a rolling upgrade, the cluster continues to operate normally. However,
|
|
any new functionality is disabled or operates in a backward compatible mode
|
|
until all nodes in the cluster are upgraded. New functionality becomes
|
|
operational once the upgrade is complete and all nodes are running the new
|
|
version. Once that has happened, there's no way to return to operating in a
|
|
backward compatible mode. Nodes running the previous major version will not be
|
|
allowed to join the fully-updated cluster.
|
|
|
|
In the unlikely case of a network malfunction during the upgrade process that
|
|
isolates all remaining old nodes from the cluster, you must take the old nodes
|
|
offline and upgrade them to enable them to join the cluster.
|
|
|
|
If you stop half or more of the master-eligible nodes all at once during the
|
|
upgrade then the cluster will become unavailable, meaning that the upgrade is
|
|
no longer a _rolling_ upgrade. If this happens, you should upgrade and restart
|
|
all of the stopped master-eligible nodes to allow the cluster to form again, as
|
|
if performing a <<restart-upgrade,full-cluster restart upgrade>>. It may also
|
|
be necessary to upgrade all of the remaining old nodes before they can join the
|
|
cluster after it re-forms.
|
|
|
|
====================================================
|