Update the create snapshot repository documentation (#3116)
* Update create snapshot repository documentation Signed-off-by: ariamarble <armarble@amazon.com> * Apply suggestions from doc review review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * made additional review changes Signed-off-by: ariamarble <armarble@amazon.com> * Apply suggestions from editorial review Co-authored-by: Nathan Bower <nbower@amazon.com> --------- Signed-off-by: ariamarble <armarble@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
This commit is contained in:
parent
7213507f9c
commit
9414701fda
_api-reference/snapshots
_tuning-your-cluster/availability-and-recovery/snapshots
|
@ -5,42 +5,65 @@ parent: Snapshot APIs
|
|||
nav_order: 1
|
||||
---
|
||||
|
||||
## Register or update snapshot repository
|
||||
# Registering or updating a snapshot repository
|
||||
|
||||
Registers a new repository to store snapshots, or updates information for an existing repository. Repositories can be of two types:
|
||||
You can register a new repository in which to store snapshots or update information for an existing repository by using the snapshots API.
|
||||
|
||||
* File system (`fs`)
|
||||
There are two types of snapshot repositories:
|
||||
|
||||
* Amazon S3 bucket (`s3`)
|
||||
* File system (`fs`): For instructions on creating an `fs` repository, see [Register repository shared file system]({{site.url}}{{site.baseurl}}/availability-and-recovery/snapshots/snapshot-restore/#shared-file-system).
|
||||
* Amazon Simple Storage Service (Amazon S3) bucket (`s3`): For instructions on creating an `s3` repository, see [Register repository Amazon S3]({{site.url}}{{site.baseurl}}/availability-and-recovery/snapshots/snapshot-restore/#amazon-s3).
|
||||
|
||||
To learn more about repositories, see [Register repository]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#register-repository).
|
||||
For instructions on creating a repository, see [Register repository]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#register-repository).
|
||||
|
||||
### Path parameters
|
||||
## Path and HTTP methods
|
||||
|
||||
```
|
||||
POST /_snapshot/my-first-repo/
|
||||
PUT /_snapshot/my-first-repo/
|
||||
```
|
||||
|
||||
## Path parameters
|
||||
|
||||
Parameter | Data type | Description
|
||||
:--- | :--- | :---
|
||||
repository | String | Repostory name. |
|
||||
repository | String | Repository name |
|
||||
|
||||
### Request fields
|
||||
## Request parameters
|
||||
|
||||
Allowable parameters depend on the repository type.
|
||||
Request parameters depend on the type of repository: `fs` or `s3`.
|
||||
|
||||
#### fs repository type
|
||||
### fs repository
|
||||
|
||||
Setting | Description
|
||||
Request field | Description
|
||||
:--- | :---
|
||||
`location` | The shared file system for snapshots. Required.
|
||||
`location` | The file system directory for snapshots, such as a mounted directory from a file server or a Samba share. Must be accessible by all nodes. Required.
|
||||
`chunk_size` | Breaks large files into chunks during snapshot operations (e.g. `64mb`, `1gb`), which is important for cloud storage providers and far less important for shared file systems. Default is `null` (unlimited). Optional.
|
||||
`compress` | Whether to compress metadata files. This setting does not affect data files, which might already be compressed, depending on your index settings. Default is `false`. Optional.
|
||||
`max_restore_bytes_per_sec` | The maximum rate at which snapshots restore. Default is 40 MB per second (`40m`). Optional.
|
||||
`max_snapshot_bytes_per_sec` | The maximum rate at which snapshots take. Default is 40 MB per second (`40m`). Optional.
|
||||
`readonly` | Whether the repository is read-only. Useful when migrating from one cluster (`"readonly": false` when registering) to another cluster (`"readonly": true` when registering). Optional.
|
||||
|
||||
#### s3 repository type
|
||||
#### Example request
|
||||
|
||||
Setting | Description
|
||||
The following example registers an `fs` repository using the local directory `/mnt/snapshots` as `location`.
|
||||
|
||||
```json
|
||||
PUT /_snapshot/my-fs-repository
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
"location": "/mnt/snapshots"
|
||||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
### s3 repository
|
||||
|
||||
Request field | Description
|
||||
:--- | :---
|
||||
`base_path` | The path within the bucket where you want to store snapshots (e.g. `my/snapshot/directory`). Optional. If not specified, snapshots are stored in the bucket root.
|
||||
`base_path` | The path within the bucket in which you want to store snapshots (for example, `my/snapshot/directory`). Optional. If not specified, snapshots are stored in the S3 bucket root.
|
||||
`bucket` | Name of the S3 bucket. Required.
|
||||
`buffer_size` | The threshold beyond which chunks (of `chunk_size`) should be broken into pieces (of `buffer_size`) and sent to S3 using a different API. Default is the smaller of two values: 100 MB or 5% of the Java heap. Valid values are between `5mb` and `5gb`. We don't recommend changing this option.
|
||||
`canned_acl` | S3 has several [canned ACLs](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) that the `repository-s3` plugin can add to objects as it creates them in S3. Default is `private`. Optional.
|
||||
|
@ -53,18 +76,15 @@ Setting | Description
|
|||
`server_side_encryption` | Whether to encrypt snapshot files in the S3 bucket. This setting uses AES-256 with S3-managed keys. See [Protecting data using server-side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html). Default is false. Optional.
|
||||
`storage_class` | Specifies the [S3 storage class](https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html) for the snapshots files. Default is `standard`. Do not use the `glacier` and `deep_archive` storage classes. Optional.
|
||||
|
||||
For the `base_path` parameter, do not enter the `s3://` prefix when entering your S3 bucket details. Only the name of the bucket is required.
|
||||
{: .note}
|
||||
|
||||
#### Example request
|
||||
|
||||
The following requests register or update a repository called `my-first-repo`. Both require a request body as described in [Request fields](#request-fields).
|
||||
|
||||
* `POST _snapshot/my-first-repo/` registers a snapshot repository.
|
||||
|
||||
* `PUT _snapshot/my-first-repo/` registers a snapshot repository if it does not exist; otherwise, updates information about the repository.
|
||||
|
||||
The following request registers a new S3 repository called `my-opensearch-repo` in an existing bucket called `my-open-search-bucket`. By default, all snapshots are stored in the `my/snapshot/directory`.
|
||||
|
||||
```json
|
||||
PUT _snapshot/my-opensearch-repo
|
||||
PUT /_snapshot/my-opensearch-repo
|
||||
{
|
||||
"type": "s3",
|
||||
"settings": {
|
||||
|
|
|
@ -53,7 +53,7 @@ Before you can take a snapshot, you have to "register" a snapshot repository. A
|
|||
1. Then register the repository using the REST API:
|
||||
|
||||
```json
|
||||
PUT _snapshot/my-fs-repository
|
||||
PUT /_snapshot/my-fs-repository
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
|
@ -61,26 +61,9 @@ Before you can take a snapshot, you have to "register" a snapshot repository. A
|
|||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
If the request is successful, the response from OpenSearch is minimal:
|
||||
|
||||
```json
|
||||
{
|
||||
"acknowledged": true
|
||||
}
|
||||
```
|
||||
|
||||
You probably only need to specify `location`, but the following table summarizes the options:
|
||||
|
||||
Request fields | Description
|
||||
:--- | :---
|
||||
`location` | The shared file system for snapshots. Required.
|
||||
`chunk_size` | Breaks large files into chunks during snapshot operations (e.g. `64mb`, `1gb`), which is important for cloud storage providers and far less important for shared file systems. Default is `null` (unlimited). Optional.
|
||||
`compress` | Whether to compress metadata files. This setting does not affect data files, which might already be compressed, depending on your index settings. Default is `false`. Optional.
|
||||
`max_restore_bytes_per_sec` | The maximum rate at which snapshots restore. Default is 40 MB per second (`40m`). Optional.
|
||||
`max_snapshot_bytes_per_sec` | The maximum rate at which snapshots take. Default is 40 MB per second (`40m`). Optional.
|
||||
`readonly` | Whether the repository is read-only. Useful when migrating from one cluster (`"readonly": false` when registering) to another cluster (`"readonly": true` when registering). Optional.
|
||||
|
||||
You will most likely not need to specify any parameters except for `location`. For allowed request parameters, see [Register or update snapshot repository API](https://opensearch.org/docs/latest/api-reference/snapshots/create-repository/).
|
||||
|
||||
### Amazon S3
|
||||
|
||||
|
@ -179,8 +162,9 @@ Request fields | Description
|
|||
1. If you changed `opensearch.yml`, you must restart each node in the cluster. Otherwise, you only need to reload secure cluster settings:
|
||||
|
||||
```
|
||||
POST _nodes/reload_secure_settings
|
||||
POST /_nodes/reload_secure_settings
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
1. Create an S3 bucket if you don't already have one. To take snapshots, you need permissions to access the bucket. The following IAM policy is an example of those permissions:
|
||||
|
||||
|
@ -203,7 +187,7 @@ Request fields | Description
|
|||
1. Register the repository using the REST API:
|
||||
|
||||
```json
|
||||
PUT _snapshot/my-s3-repository
|
||||
PUT /_snapshot/my-s3-repository
|
||||
{
|
||||
"type": "s3",
|
||||
"settings": {
|
||||
|
@ -212,24 +196,9 @@ Request fields | Description
|
|||
}
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
You probably don't need to specify anything but `bucket` and `base_path`, but the following table summarizes the options:
|
||||
|
||||
Request fields | Description
|
||||
:--- | :---
|
||||
`base_path` | The path within the bucket where you want to store snapshots (e.g. `my/snapshot/directory`). Optional. If not specified, snapshots are stored in the bucket root.
|
||||
`bucket` | Name of the S3 bucket. Required.
|
||||
`buffer_size` | The threshold beyond which chunks (of `chunk_size`) should be broken into pieces (of `buffer_size`) and sent to S3 using a different API. Default is the smaller of two values: 100 MB or 5% of the Java heap. Valid values are between `5mb` and `5gb`. We don't recommend changing this option.
|
||||
`canned_acl` | S3 has several [canned ACLs](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) that the `repository-s3` plugin can add to objects as it creates them in S3. Default is `private`. Optional.
|
||||
`chunk_size` | Breaks files into chunks during snapshot operations (e.g. `64mb`, `1gb`), which is important for cloud storage providers and far less important for shared file systems. Default is `1gb`. Optional.
|
||||
`client` | When specifying client settings (e.g. `s3.client.default.access_key`), you can use a string other than `default` (e.g. `s3.client.backup-role.access_key`). If you used an alternate name, change this value to match. Default and recommended value is `default`. Optional.
|
||||
`compress` | Whether to compress metadata files. This setting does not affect data files, which might already be compressed, depending on your index settings. Default is `false`. Optional.
|
||||
`max_restore_bytes_per_sec` | The maximum rate at which snapshots restore. Default is 40 MB per second (`40m`). Optional.
|
||||
`max_snapshot_bytes_per_sec` | The maximum rate at which snapshots take. Default is 40 MB per second (`40m`). Optional.
|
||||
`readonly` | Whether the repository is read-only. Useful when migrating from one cluster (`"readonly": false` when registering) to another cluster (`"readonly": true` when registering). Optional.
|
||||
`server_side_encryption` | Whether to encrypt snapshot files in the S3 bucket. This setting uses AES-256 with S3-managed keys. See [Protecting data using server-side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html). Default is false. Optional.
|
||||
`storage_class` | Specifies the [S3 storage class](https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html) for the snapshots files. Default is `standard`. Do not use the `glacier` and `deep_archive` storage classes. Optional.
|
||||
|
||||
You will most likely not need to specify any parameters except for `bucket` and `base_path`. For allowed request parameters, see [Register or update snapshot repository API](https://opensearch.org/docs/latest/api-reference/snapshots/create-repository/).
|
||||
|
||||
## Take snapshots
|
||||
|
||||
|
@ -241,13 +210,14 @@ You specify two pieces of information when you create a snapshot:
|
|||
The following snapshot includes all indices and the cluster state:
|
||||
|
||||
```json
|
||||
PUT _snapshot/my-repository/1
|
||||
PUT /_snapshot/my-repository/1
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
You can also add a request body to include or exclude certain indices or specify other settings:
|
||||
|
||||
```json
|
||||
PUT _snapshot/my-repository/2
|
||||
PUT /_snapshot/my-repository/2
|
||||
{
|
||||
"indices": "opensearch-dashboards*,my-index*,-my-index-2016",
|
||||
"ignore_unavailable": true,
|
||||
|
@ -255,6 +225,7 @@ PUT _snapshot/my-repository/2
|
|||
"partial": false
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Request fields | Description
|
||||
:--- | :---
|
||||
|
@ -266,7 +237,7 @@ Request fields | Description
|
|||
If you request the snapshot immediately after taking it, you might see something like this:
|
||||
|
||||
```json
|
||||
GET _snapshot/my-repository/2
|
||||
GET /_snapshot/my-repository/2
|
||||
{
|
||||
"snapshots": [{
|
||||
"snapshot": "2",
|
||||
|
@ -283,12 +254,14 @@ GET _snapshot/my-repository/2
|
|||
}]
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Note that the snapshot is still in progress. If you want to wait for the snapshot to finish before continuing, add the `wait_for_completion` parameter to your request. Snapshots can take a while to complete, so consider whether or not this option fits your use case:
|
||||
|
||||
```
|
||||
PUT _snapshot/my-repository/3?wait_for_completion=true
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Snapshots have the following states:
|
||||
|
||||
|
@ -303,8 +276,9 @@ INCOMPATIBLE | The snapshot is incompatible with the version of OpenSearch runni
|
|||
You can't take a snapshot if one is currently in progress. To check the status:
|
||||
|
||||
```
|
||||
GET _snapshot/_status
|
||||
GET /_snapshot/_status
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
|
||||
## Restore snapshots
|
||||
|
@ -312,25 +286,28 @@ GET _snapshot/_status
|
|||
The first step in restoring a snapshot is retrieving existing snapshots. To see all snapshot repositories:
|
||||
|
||||
```
|
||||
GET _snapshot/_all
|
||||
GET /_snapshot/_all
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
To see all snapshots in a repository:
|
||||
|
||||
```
|
||||
GET _snapshot/my-repository/_all
|
||||
GET /_snapshot/my-repository/_all
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Then restore a snapshot:
|
||||
|
||||
```
|
||||
POST _snapshot/my-repository/2/_restore
|
||||
POST /_snapshot/my-repository/2/_restore
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Just like when taking a snapshot, you can add a request body to include or exclude certain indices or specify some other settings:
|
||||
|
||||
```json
|
||||
POST _snapshot/my-repository/2/_restore
|
||||
POST /_snapshot/my-repository/2/_restore
|
||||
{
|
||||
"indices": "opensearch-dashboards*,my-index*",
|
||||
"ignore_unavailable": true,
|
||||
|
@ -347,8 +324,9 @@ POST _snapshot/my-repository/2/_restore
|
|||
]
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
Request fields | Description
|
||||
Request parameters | Description
|
||||
:--- | :---
|
||||
`indices` | The indices you want to restore. You can use `,` to create a list of indices, `*` to specify an index pattern, and `-` to exclude certain indices. Don't put spaces between items. Default is all indices.
|
||||
`ignore_unavailable` | If an index from the `indices` list doesn't exist, whether to ignore it rather than fail the restore operation. Default is false.
|
||||
|
@ -385,18 +363,20 @@ If you're using the security plugin, snapshots have some additional restrictions
|
|||
If a snapshot contains global state, you must exclude it when performing the restore. If your snapshot also contains the `.opendistro_security` index, either exclude it or list all the other indices you want to include:
|
||||
|
||||
```json
|
||||
POST _snapshot/my-repository/3/_restore
|
||||
POST /_snapshot/my-repository/3/_restore
|
||||
{
|
||||
"indices": "-.opendistro_security",
|
||||
"include_global_state": false
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
The `.opendistro_security` index contains sensitive data, so we recommend excluding it when you take a snapshot. If you do need to restore the index from a snapshot, you must include an admin certificate in the request:
|
||||
|
||||
```bash
|
||||
curl -k --cert ./kirk.pem --key ./kirk-key.pem -XPOST 'https://localhost:9200/_snapshot/my-repository/3/_restore?pretty'
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
We strongly recommend against restoring `.opendistro_security` using an admin certificate because doing so can alter the security posture of the entire cluster. See [A word of caution]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin/#a-word-of-caution) for a recommended process to back up and restore your security plugin configuration.
|
||||
{: .warning}
|
Loading…
Reference in New Issue