[DOCS] Adding new page for restore snapshot API (#59937) (#60055)

* Adding new page for restore snapshot API.

* Improving test cases, lots of edits, and streamlining content.

* Incorporating review suggestions and feedback.

* Specify `index alias` vs `alias`

* Change parameter order

* Provide clarity around regular expression

* Add link to SLM parameters

* Split sentences in example

* Adding link to master node page.
This commit is contained in:
Adam Locke 2020-07-22 12:08:55 -04:00 committed by GitHub
parent 46d33b1586
commit 0a73225cd8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 272 additions and 54 deletions

View File

@ -95,7 +95,7 @@ indices.
Below is a list of all _dynamic_ index settings that are not associated with any
specific index module:
[[dynamic-index-number-of-replicas]]
`index.number_of_replicas`::
The number of replicas each primary shard has. Defaults to 1.

View File

@ -0,0 +1,210 @@
[[restore-snapshot-api]]
=== Restore snapshot API
++++
<titleabbrev>Restore snapshot</titleabbrev>
++++
Restores a <<snapshot-restore,snapshot>> of a cluster or specified data streams and indices.
////
[source,console]
----
PUT /_snapshot/my_repository
{
"type": "fs",
"settings": {
"location": "my_backup_location"
}
}
PUT /_snapshot/my_repository/my_snapshot?wait_for_completion=true
PUT /index_1
PUT /index_2
PUT /index_3
PUT /index_4
PUT _snapshot/my_repository/snapshot_2?wait_for_completion=true
{
"indices": "index_3,index_4",
"ignore_unavailable": true,
"include_global_state": false,
"metadata": {
"taken_by": "Elastic Machine",
"taken_because": "backup testing"
}
}
POST /index_1/_close
POST /index_2/_close
POST /index_3/_close
POST /index_4/_close
----
// TESTSETUP
////
[source,console]
----
POST /_snapshot/my_repository/my_snapshot/_restore
----
// TEST[s/_restore/_restore?wait_for_completion=true/]
[[restore-snapshot-api-request]]
==== {api-request-title}
`POST /_snapshot/<repository>/<snapshot>`
[[restore-snapshot-api-desc]]
==== {api-description-title}
Use the restore snapshot API to restore a snapshot of a cluster, including all data streams and indices in the snapshot. If you do not want to restore the entire snapshot, you can select specific data streams or indices to restore.
NOTE: You cannot restore a data stream if a stream with the same name already
exists.
You can run the restore operation on a cluster that contains an elected
<<master-node,master node>> and has data nodes with enough capacity to accommodate the snapshot
you are restoring. Existing indices can only be restored if they are
<<indices-close,closed>> and have the same number of shards as the indices in
the snapshot. The restore operation automatically opens restored indices if
they were closed and creates new indices if they do not exist in the cluster.
If a data stream is restored, its backing indices are also restored.
Alternatively, you can restore individual backing indices without restoring an
entire data stream. If you restore individual backing indices, they are not
automatically added to any existing data stream. For example, if only the
`.ds-logs-00003` backing index is restored from a snapshot, it is not
automatically added to the existing `logs` data stream.
include::{es-ref-dir}/snapshot-restore/restore-snapshot.asciidoc[tag=index-settings-data-stream-warning]
[[restore-snapshot-api-path-params]]
==== {api-path-parms-title}
`<repository>`::
(Required, string)
Name of the repository to restore a snapshot from.
`<snapshot>`::
(Required, string)
Name of the snapshot to restore.
[role="child-attributes"]
[[restore-snapshot-api-request-body]]
==== {api-request-body-title}
`ignore_index_settings`::
(Optional, string)
A comma-separated list of index settings that should not be restored from a snapshot.
`include_aliases`::
(Optional, boolean)
If `true`, index aliases from the original snapshot are restored.
Defaults to `true`.
+
If `false`, prevents aliases from being restored together with associated
indices.
[[restore-snapshot-api-include-global-state]]
`include_global_state`::
+
--
(Optional, boolean)
If `false`, the cluster state is not restored. Defaults to `false`.
If `true`, the current cluster state is included in the restore operation.
The cluster state includes:
* Persistent cluster settings
* Index templates
* Legacy index templates
* Ingest pipelines
* {ilm-init} lifecycle policies
--
+
IMPORTANT: By default, the entire restore operation will fail if one or more indices included in the snapshot do not have all primary shards available. You can change this behavior by setting <<restore-snapshot-api-partial,`partial`>> to `true`.
[[restore-snapshot-api-index-settings]]
`index_settings`::
(Optional, string)
A comma-separated list of settings to add or change in all restored indices. Use this parameter to override index settings when restoring snapshots.
+
For data streams, these index settings are applied to the restored backing
indices.
+
For more information regarding all the different index-level settings
that you can specify, see
<<index-modules,index modules>>.
`indices`::
(Optional, string)
A comma-separated list of data streams and indices to restore from the snapshot.
<<multi-index,Multi-index syntax>> is supported.
+
By default, a restore operation includes all data streams and indices in the snapshot. If this
argument is provided, the restore operation only includes the specified data streams and indices.
[[restore-snapshot-api-partial]]
`partial`::
(Optional, boolean)
If `false`, the entire restore operation will fail if one or more indices included in the snapshot do not have all primary shards available. Defaults to `false`.
+
If `true`, allows restoring a partial snapshot of indices with unavailable shards. Only shards that were successfully included in the snapshot
will be restored. All missing shards will be recreated as empty.
[[restore-snapshot-api-rename-pattern]]
`rename_pattern`::
(Optional, string)
Defines a rename pattern to apply to restored data streams and indices. Data streams and indices matching the rename pattern will be renamed according to
<<restore-snapshot-api-rename-replacement,`rename_replacement`>>.
+
The rename pattern is applied as defined by the regular expression that
supports referencing the original text, according to the https://docs.oracle.com/javase/8/docs/api/java/util/regex/Matcher.html#appendReplacement-java.lang.StringBuffer-java.lang.String-[`appendReplacement`] logic.
+
The request will fail if two or more data streams or indices will be renamed into the same name.
+
include::{es-ref-dir}/snapshot-restore/restore-snapshot.asciidoc[tag=rename-restored-data-stream-tag]
[[restore-snapshot-api-rename-replacement]]
`rename_replacement`::
(Optional, string)
Defines the rename replacement string. See <<restore-snapshot-api-rename-pattern,`rename_pattern`>> for more information.
`wait_for_completion`::
(Optional, boolean)
If `false`, the request returns a response when the restore operation initializes.
Defaults to `false`.
+
If `true`, the request returns a response when the restore operation
completes.
[[restore-snapshot-api-example]]
==== {api-examples-title}
The following request restores `index_1` and `index_2` from `snapshot_2`. The `rename_pattern` and `rename_replacement` parameters indicate any index matching the regular expression `index_(.+)` will be renamed using the pattern `restored_index_$1` when restored.
For example, `index_1` will be renamed to `restored_index_1`. `index_2` will be renamed to `restored_index_2`.
[source,console]
----
POST /_snapshot/my_repository/snapshot_2/_restore?wait_for_completion=true
{
"indices": "index_1,index_2",
"ignore_unavailable": true,
"include_global_state": false,
"rename_pattern": "index_(.+)",
"rename_replacement": "restored_index_$1",
"include_aliases": false
}
----
The API returns an acknowledgement if the request succeeds. If the request encounters errors, the response indicates any issues found, such as
open indices that are blocking the restore operation from completing.

View File

@ -6,12 +6,6 @@ backups, and restore snapshots to a running cluster.
For more information, see <<snapshot-restore>>.
[NOTE]
====
We are working on including more snapshot and restore APIs in this section. Some
content may not be included yet.
====
[discrete]
[[snapshot-restore-repo-apis]]
=== Snapshot repository management APIs
@ -27,6 +21,7 @@ content may not be included yet.
* <<create-snapshot-api,Create snapshot>>
* <<get-snapshot-api,Get snapshot>>
* <<get-snapshot-status-api,Get snapshot status>>
* <<restore-snapshot-api,Restore snapshot>>
* <<delete-snapshot-api,Delete snapshot>>
include::put-repo-api.asciidoc[]
@ -37,4 +32,5 @@ include::clean-up-repo-api.asciidoc[]
include::create-snapshot-api.asciidoc[]
include::get-snapshot-api.asciidoc[]
include::get-snapshot-status-api.asciidoc[]
include::restore-snapshot-api.asciidoc[]
include::delete-snapshot-api.asciidoc[]

View File

@ -10,10 +10,14 @@ You can take snapshots of an entire cluster, including all its data streams and
indices. You can also take snapshots of only specific data streams or indices in
the cluster.
You must
<<snapshots-register-repository, register a snapshot repository>>
before you can <<snapshots-take-snapshot, create snapshots>>.
Snapshots can be stored in either local or remote repositories.
Remote repositories can reside on Amazon S3, HDFS, Microsoft Azure,
Google Cloud Storage,
and other platforms supported by a repository plugin.
and other platforms supported by a {plugins}/repository.html[repository plugin].
Snapshots are incremental: each snapshot only stores data that
is not part of an earlier snapshot.
@ -21,16 +25,14 @@ This enables you to take frequent snapshots with minimal overhead.
// end::snapshot-intro[]
// tag::restore-intro[]
You can restore snapshots to a running cluster with the <<snapshots-restore-snapshot,restore API>>.
By default, all data streams and indices in the snapshot are restored.
You can <<snapshots-restore-snapshot,restore snapshots>> to a running cluster, which includes all data streams and indices in the snapshot
by default.
However, you can choose to restore only the cluster state or specific data
streams or indices from a snapshot.
// end::restore-intro[]
You must <<snapshots-register-repository, register a snapshot repository>>
before you can <<snapshots-take-snapshot, take snapshots>>.
You can use <<getting-started-snapshot-lifecycle-management, snapshot lifecycle management>>
You can use
<<getting-started-snapshot-lifecycle-management, {slm}>>
to automatically take and manage snapshots.
// tag::backup-warning[]

View File

@ -7,19 +7,23 @@
[[snapshots-register-repository-description]]
// tag::snapshots-register-repository-tag[]
You must register a snapshot repository before you can perform snapshot and
restore operations. We recommend creating a new snapshot repository for each
restore operations. Use the <<put-snapshot-repo-api,put snapshot repository API>> to register or update a snapshot repository. We recommend creating a new snapshot repository for each
major version. The valid repository settings depend on the repository type.
If you register same snapshot repository with multiple clusters, only
If you register the same snapshot repository with multiple clusters, only
one cluster should have write access to the repository. All other clusters
connected to that repository should set the repository to `readonly` mode.
// end::snapshots-register-repository-tag[]
IMPORTANT: The snapshot format can change across major versions, so if you have
clusters on different versions trying to write the same repository, snapshots
[IMPORTANT]
====
The snapshot format can change across major versions, so if you have
clusters on different versions trying to write to the same repository, snapshots
written by one version may not be visible to the other and the repository could
be corrupted. While setting the repository to `readonly` on all but one of the
clusters should work with multiple clusters differing by one major version, it
is not a supported configuration.
====
[source,console]
-----------------------------------
@ -33,14 +37,14 @@ PUT /_snapshot/my_backup
-----------------------------------
// TESTSETUP
To retrieve information about a registered repository, use a GET request:
Use the <<get-snapshot-api,get snapshot API>> to retrieve information about a registered repository:
[source,console]
-----------------------------------
GET /_snapshot/my_backup
-----------------------------------
which returns:
This request returns the following response:
[source,console-result]
-----------------------------------
@ -55,7 +59,7 @@ which returns:
-----------------------------------
To retrieve information about multiple repositories, specify a comma-delimited
list of repositories. You can also use the * wildcard when
list of repositories. You can also use a wildcard (`*`) when
specifying repository names. For example, the following request retrieves
information about all of the snapshot repositories that start with `repo` or
contain `backup`:
@ -66,22 +70,22 @@ GET /_snapshot/repo*,*backup*
-----------------------------------
To retrieve information about all registered snapshot repositories, omit the
repository name or specify `_all`:
repository name:
[source,console]
-----------------------------------
GET /_snapshot
-----------------------------------
or
Alternatively, you can specify `_all`:
[source,console]
-----------------------------------
GET /_snapshot/_all
-----------------------------------
You can unregister a repository using the <<delete-snapshot-repo-api,delete
snapshot repository API>>:
You can unregister a repository using the
<<delete-snapshot-repo-api,delete snapshot repository API>>:
[source,console]
-----------------------------------
@ -151,7 +155,6 @@ PUT /_snapshot/my_fs_backup
The following settings are supported:
[horizontal]
`location`:: Location of the snapshots. Mandatory.
`compress`:: Turns on compression of the snapshot files. Compression is applied only to metadata files (index mapping and settings). Data files are not compressed. Defaults to `true`.
`chunk_size`:: Big files can be broken down into chunks during snapshotting if needed. Specify the chunk size as a value and

View File

@ -18,18 +18,19 @@ PUT /_snapshot/my_backup/snapshot_1?wait_for_completion=true
////
A snapshot can be restored using the following command:
Use the <<restore-snapshot-api,restore snapshot API>> to restore
a snapshot of a cluster or specified data streams and indices:
[source,console]
-----------------------------------
POST /_snapshot/my_backup/snapshot_1/_restore
-----------------------------------
// TEST[s/_restore/_restore?wait_for_completion=true/]
By default, all data streams and indices in the snapshot are restored, but the cluster state is
*not* restored. It's possible to select specific data streams or indices that should be restored as well
as to allow the global cluster state from being restored by using `indices` and
`include_global_state` options in the restore request body. The list
supports <<multi-index,multi-target syntax>>.
*not* restored. Use the `indices` parameter to restore only specific data streams or indices. This parameter
supports <<multi-index,multi-target syntax>>. To include the global cluster state, set
`include_global_state` to `true` in the restore request body.
[WARNING]
====
@ -52,10 +53,10 @@ If no index template matches a data stream, the stream cannot
The `rename_pattern`
and `rename_replacement` options can be also used to rename data streams and indices on restore
using regular expression that supports referencing the original text as
explained
http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#appendReplacement(java.lang.StringBuffer,%20java.lang.String)[here].
using regular expression that supports referencing the original text, according to the https://docs.oracle.com/javase/8/docs/api/java/util/regex/Matcher.html#appendReplacement-java.lang.StringBuffer-java.lang.String-[`appendReplacement`] logic.
[[rename-restored-data-stream]]
// tag::rename-restored-data-stream-tag[]
If you rename a restored data stream, its backing indices are also
renamed. For example, if you rename the `logs` data stream to `restored-logs`,
the backing index `.ds-logs-000005` is renamed to `.ds-restored-logs-000005`.
@ -66,6 +67,7 @@ If you rename a restored stream, ensure an index template matches the new stream
name. If no index template matches the stream, it cannot
<<manually-roll-over-a-data-stream,roll over>> or create new backing indices.
====
// end::rename-restored-data-stream-tag[]
Set `include_aliases` to `false` to prevent aliases from being restored together
with associated indices
@ -92,8 +94,8 @@ pipelines, and {ilm-init} policies are restored into the current cluster. This
overwrites any existing cluster settings, templates, pipelines and {ilm-init}
policies whose names match those in the snapshot.
The restore operation can be performed on a functioning cluster. However, an
existing index can be only restored if it's <<indices-open-close,closed>> and
The restore operation must be performed on a functioning cluster. However, an
existing index can be only restored if it's <<indices-close,closed>> and
has the same number of shards as the index in the snapshot. The restore
operation automatically opens restored indices if they were closed and creates
new indices if they didn't exist in the cluster.
@ -110,7 +112,7 @@ added to any existing data streams. For example, if only the `.ds-logs-000003`
backing index is restored from a snapshot, it is not automatically added to the
existing `logs` data stream.
[float]
[discrete]
=== Partial restore
By default, the entire restore operation will fail if one or more indices or backing indices participating in the operation don't have
@ -119,11 +121,13 @@ restore such indices by setting `partial` to `true`. Please note, that only succ
restored in this case and all missing shards will be recreated empty.
[float]
[discrete]
=== Changing index settings during restore
Most of index settings can be overridden during the restore process. For example, the following command will restore
the index `index_1` without creating any replicas while switching back to default refresh interval:
Use the <<restore-snapshot-api-index-settings,`index_settings`>> parameter
to override index settings during the restore process. For example, the
following request will restore the index `index_1` without creating any
replicas while switching back to the default refresh interval:
[source,console]
-----------------------------------
@ -141,11 +145,12 @@ POST /_snapshot/my_backup/snapshot_1/_restore
-----------------------------------
// TEST[continued]
Please note, that some settings such as `index.number_of_shards` cannot be changed during restore operation.
NOTE: Some settings such as `index.number_of_shards` cannot be changed during the restore operation.
For data streams, these index settings are applied to the restored backing
indices.
// tag::index-settings-data-stream-warning[]
[IMPORTANT]
====
The `index_settings` and `ignore_index_settings` parameters affect
@ -157,25 +162,27 @@ If you change index settings during a restore, we recommend you make similar
changes in the stream's matching index template. This ensures new backing
indices created for the stream use the same index settings.
====
// end::index-settings-data-stream-warning[]
[float]
[discrete]
=== Restoring to a different cluster
The information stored in a snapshot is not tied to a particular cluster or a cluster name. Therefore it's possible to
restore a snapshot made from one cluster into another cluster. All that is required is registering the repository
containing the snapshot in the new cluster and starting the restore process. The new cluster doesn't have to have the
same size or topology. However, the version of the new cluster should be the same or newer (only 1 major version newer) than the cluster that was used to create the snapshot. For example, you can restore a 1.x snapshot to a 2.x cluster, but not a 1.x snapshot to a 5.x cluster.
The information stored in a snapshot is not tied to a particular cluster or a cluster name. Therefore, it's possible to
restore a snapshot created from one cluster into another cluster by registering the repository that contains the snapshot in the new cluster and starting the restore process.
If the new cluster has a smaller size additional considerations should be made. First of all it's necessary to make sure
that new cluster have enough capacity to store all data streams and indices in the snapshot. It's possible to change index settings
during restore to reduce the number of replicas, which can help with restoring snapshots into smaller cluster. It's also
possible to select only subset of the data streams or indices using the `indices` parameter.
The topology of the new cluster does not have to match the cluster where the snapshot was created. However, the version of the new cluster must be the same or only one major version newer than the cluster that was used to create the snapshot. For example, you can restore a 5.x snapshot to a 6.x cluster, but not a 5.x snapshot to a 7.x cluster.
If the new cluster has a smaller capacity, consider the following changes:
* Ensure that the new cluster has enough capacity to store all data streams and indices in the snapshot.
* Change the index settings during the restore operation to reduce the
<<dynamic-index-number-of-replicas,number of replicas>>.
* Use the `indices` parameter to choose only specific data streams or indices to restore.
If indices or backing indices in the original cluster were assigned to particular nodes using
<<shard-allocation-filtering,shard allocation filtering>>, the same rules will be enforced in the new cluster. Therefore
if the new cluster doesn't contain nodes with appropriate attributes that a restored index can be allocated on, such
index will not be successfully restored unless these index allocation settings are changed during restore operation.
<<shard-allocation-filtering,shard allocation filtering>>, the same rules will be enforced in the new cluster. If the new cluster does not contain nodes with appropriate attributes that a restored index can be allocated on, the
index will not be successfully restored unless these index allocation settings are changed during the restore operation.
The restore operation also checks that restored persistent settings are compatible with the current cluster to avoid accidentally
restoring incompatible settings. If you need to restore a snapshot with incompatible persistent settings, try restoring it without
the global cluster state.
the <<restore-snapshot-api-include-global-state,global cluster state>>.