Update the create snapshot repository documentation ()

* Update create snapshot repository documentation

Signed-off-by: ariamarble <armarble@amazon.com>

* Apply suggestions from doc review review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* made additional review changes

Signed-off-by: ariamarble <armarble@amazon.com>

* Apply suggestions from editorial review

Co-authored-by: Nathan Bower <nbower@amazon.com>

---------

Signed-off-by: ariamarble <armarble@amazon.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
This commit is contained in:
Aria Marble 2023-03-01 14:52:22 -08:00 committed by GitHub
parent 7213507f9c
commit 9414701fda
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 71 additions and 71 deletions
_api-reference/snapshots
_tuning-your-cluster/availability-and-recovery/snapshots

View File

@ -5,42 +5,65 @@ parent: Snapshot APIs
nav_order: 1
---
## Register or update snapshot repository
# Registering or updating a snapshot repository
Registers a new repository to store snapshots, or updates information for an existing repository. Repositories can be of two types:
You can register a new repository in which to store snapshots or update information for an existing repository by using the snapshots API.
* File system (`fs`)
There are two types of snapshot repositories:
* Amazon S3 bucket (`s3`)
* File system (`fs`): For instructions on creating an `fs` repository, see [Register repository shared file system]({{site.url}}{{site.baseurl}}/availability-and-recovery/snapshots/snapshot-restore/#shared-file-system).
* Amazon Simple Storage Service (Amazon S3) bucket (`s3`): For instructions on creating an `s3` repository, see [Register repository Amazon S3]({{site.url}}{{site.baseurl}}/availability-and-recovery/snapshots/snapshot-restore/#amazon-s3).
To learn more about repositories, see [Register repository]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#register-repository).
For instructions on creating a repository, see [Register repository]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#register-repository).
### Path parameters
## Path and HTTP methods
```
POST /_snapshot/my-first-repo/
PUT /_snapshot/my-first-repo/
```
## Path parameters
Parameter | Data type | Description
:--- | :--- | :---
repository | String | Repostory name. |
repository | String | Repository name |
### Request fields
## Request parameters
Allowable parameters depend on the repository type.
Request parameters depend on the type of repository: `fs` or `s3`.
#### fs repository type
### fs repository
Setting | Description
Request field | Description
:--- | :---
`location` | The shared file system for snapshots. Required.
`location` | The file system directory for snapshots, such as a mounted directory from a file server or a Samba share. Must be accessible by all nodes. Required.
`chunk_size` | Breaks large files into chunks during snapshot operations (e.g. `64mb`, `1gb`), which is important for cloud storage providers and far less important for shared file systems. Default is `null` (unlimited). Optional.
`compress` | Whether to compress metadata files. This setting does not affect data files, which might already be compressed, depending on your index settings. Default is `false`. Optional.
`max_restore_bytes_per_sec` | The maximum rate at which snapshots restore. Default is 40 MB per second (`40m`). Optional.
`max_snapshot_bytes_per_sec` | The maximum rate at which snapshots take. Default is 40 MB per second (`40m`). Optional.
`readonly` | Whether the repository is read-only. Useful when migrating from one cluster (`"readonly": false` when registering) to another cluster (`"readonly": true` when registering). Optional.
#### s3 repository type
#### Example request
Setting | Description
The following example registers an `fs` repository using the local directory `/mnt/snapshots` as `location`.
```json
PUT /_snapshot/my-fs-repository
{
"type": "fs",
"settings": {
"location": "/mnt/snapshots"
}
}
```
{% include copy-curl.html %}
### s3 repository
Request field | Description
:--- | :---
`base_path` | The path within the bucket where you want to store snapshots (e.g. `my/snapshot/directory`). Optional. If not specified, snapshots are stored in the bucket root.
`base_path` | The path within the bucket in which you want to store snapshots (for example, `my/snapshot/directory`). Optional. If not specified, snapshots are stored in the S3 bucket root.
`bucket` | Name of the S3 bucket. Required.
`buffer_size` | The threshold beyond which chunks (of `chunk_size`) should be broken into pieces (of `buffer_size`) and sent to S3 using a different API. Default is the smaller of two values: 100 MB or 5% of the Java heap. Valid values are between `5mb` and `5gb`. We don't recommend changing this option.
`canned_acl` | S3 has several [canned ACLs](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) that the `repository-s3` plugin can add to objects as it creates them in S3. Default is `private`. Optional.
@ -53,18 +76,15 @@ Setting | Description
`server_side_encryption` | Whether to encrypt snapshot files in the S3 bucket. This setting uses AES-256 with S3-managed keys. See [Protecting data using server-side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html). Default is false. Optional.
`storage_class` | Specifies the [S3 storage class](https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html) for the snapshots files. Default is `standard`. Do not use the `glacier` and `deep_archive` storage classes. Optional.
For the `base_path` parameter, do not enter the `s3://` prefix when entering your S3 bucket details. Only the name of the bucket is required.
{: .note}
#### Example request
The following requests register or update a repository called `my-first-repo`. Both require a request body as described in [Request fields](#request-fields).
* `POST _snapshot/my-first-repo/` registers a snapshot repository.
* `PUT _snapshot/my-first-repo/` registers a snapshot repository if it does not exist; otherwise, updates information about the repository.
The following request registers a new S3 repository called `my-opensearch-repo` in an existing bucket called `my-open-search-bucket`. By default, all snapshots are stored in the `my/snapshot/directory`.
```json
PUT _snapshot/my-opensearch-repo
PUT /_snapshot/my-opensearch-repo
{
"type": "s3",
"settings": {

View File

@ -53,7 +53,7 @@ Before you can take a snapshot, you have to "register" a snapshot repository. A
1. Then register the repository using the REST API:
```json
PUT _snapshot/my-fs-repository
PUT /_snapshot/my-fs-repository
{
"type": "fs",
"settings": {
@ -61,26 +61,9 @@ Before you can take a snapshot, you have to "register" a snapshot repository. A
}
}
```
{% include copy-curl.html %}
If the request is successful, the response from OpenSearch is minimal:
```json
{
"acknowledged": true
}
```
You probably only need to specify `location`, but the following table summarizes the options:
Request fields | Description
:--- | :---
`location` | The shared file system for snapshots. Required.
`chunk_size` | Breaks large files into chunks during snapshot operations (e.g. `64mb`, `1gb`), which is important for cloud storage providers and far less important for shared file systems. Default is `null` (unlimited). Optional.
`compress` | Whether to compress metadata files. This setting does not affect data files, which might already be compressed, depending on your index settings. Default is `false`. Optional.
`max_restore_bytes_per_sec` | The maximum rate at which snapshots restore. Default is 40 MB per second (`40m`). Optional.
`max_snapshot_bytes_per_sec` | The maximum rate at which snapshots take. Default is 40 MB per second (`40m`). Optional.
`readonly` | Whether the repository is read-only. Useful when migrating from one cluster (`"readonly": false` when registering) to another cluster (`"readonly": true` when registering). Optional.
You will most likely not need to specify any parameters except for `location`. For allowed request parameters, see [Register or update snapshot repository API](https://opensearch.org/docs/latest/api-reference/snapshots/create-repository/).
### Amazon S3
@ -179,8 +162,9 @@ Request fields | Description
1. If you changed `opensearch.yml`, you must restart each node in the cluster. Otherwise, you only need to reload secure cluster settings:
```
POST _nodes/reload_secure_settings
POST /_nodes/reload_secure_settings
```
{% include copy-curl.html %}
1. Create an S3 bucket if you don't already have one. To take snapshots, you need permissions to access the bucket. The following IAM policy is an example of those permissions:
@ -203,7 +187,7 @@ Request fields | Description
1. Register the repository using the REST API:
```json
PUT _snapshot/my-s3-repository
PUT /_snapshot/my-s3-repository
{
"type": "s3",
"settings": {
@ -212,24 +196,9 @@ Request fields | Description
}
}
```
{% include copy-curl.html %}
You probably don't need to specify anything but `bucket` and `base_path`, but the following table summarizes the options:
Request fields | Description
:--- | :---
`base_path` | The path within the bucket where you want to store snapshots (e.g. `my/snapshot/directory`). Optional. If not specified, snapshots are stored in the bucket root.
`bucket` | Name of the S3 bucket. Required.
`buffer_size` | The threshold beyond which chunks (of `chunk_size`) should be broken into pieces (of `buffer_size`) and sent to S3 using a different API. Default is the smaller of two values: 100 MB or 5% of the Java heap. Valid values are between `5mb` and `5gb`. We don't recommend changing this option.
`canned_acl` | S3 has several [canned ACLs](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) that the `repository-s3` plugin can add to objects as it creates them in S3. Default is `private`. Optional.
`chunk_size` | Breaks files into chunks during snapshot operations (e.g. `64mb`, `1gb`), which is important for cloud storage providers and far less important for shared file systems. Default is `1gb`. Optional.
`client` | When specifying client settings (e.g. `s3.client.default.access_key`), you can use a string other than `default` (e.g. `s3.client.backup-role.access_key`). If you used an alternate name, change this value to match. Default and recommended value is `default`. Optional.
`compress` | Whether to compress metadata files. This setting does not affect data files, which might already be compressed, depending on your index settings. Default is `false`. Optional.
`max_restore_bytes_per_sec` | The maximum rate at which snapshots restore. Default is 40 MB per second (`40m`). Optional.
`max_snapshot_bytes_per_sec` | The maximum rate at which snapshots take. Default is 40 MB per second (`40m`). Optional.
`readonly` | Whether the repository is read-only. Useful when migrating from one cluster (`"readonly": false` when registering) to another cluster (`"readonly": true` when registering). Optional.
`server_side_encryption` | Whether to encrypt snapshot files in the S3 bucket. This setting uses AES-256 with S3-managed keys. See [Protecting data using server-side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html). Default is false. Optional.
`storage_class` | Specifies the [S3 storage class](https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html) for the snapshots files. Default is `standard`. Do not use the `glacier` and `deep_archive` storage classes. Optional.
You will most likely not need to specify any parameters except for `bucket` and `base_path`. For allowed request parameters, see [Register or update snapshot repository API](https://opensearch.org/docs/latest/api-reference/snapshots/create-repository/).
## Take snapshots
@ -241,13 +210,14 @@ You specify two pieces of information when you create a snapshot:
The following snapshot includes all indices and the cluster state:
```json
PUT _snapshot/my-repository/1
PUT /_snapshot/my-repository/1
```
{% include copy-curl.html %}
You can also add a request body to include or exclude certain indices or specify other settings:
```json
PUT _snapshot/my-repository/2
PUT /_snapshot/my-repository/2
{
"indices": "opensearch-dashboards*,my-index*,-my-index-2016",
"ignore_unavailable": true,
@ -255,6 +225,7 @@ PUT _snapshot/my-repository/2
"partial": false
}
```
{% include copy-curl.html %}
Request fields | Description
:--- | :---
@ -266,7 +237,7 @@ Request fields | Description
If you request the snapshot immediately after taking it, you might see something like this:
```json
GET _snapshot/my-repository/2
GET /_snapshot/my-repository/2
{
"snapshots": [{
"snapshot": "2",
@ -283,12 +254,14 @@ GET _snapshot/my-repository/2
}]
}
```
{% include copy-curl.html %}
Note that the snapshot is still in progress. If you want to wait for the snapshot to finish before continuing, add the `wait_for_completion` parameter to your request. Snapshots can take a while to complete, so consider whether or not this option fits your use case:
```
PUT _snapshot/my-repository/3?wait_for_completion=true
```
{% include copy-curl.html %}
Snapshots have the following states:
@ -303,8 +276,9 @@ INCOMPATIBLE | The snapshot is incompatible with the version of OpenSearch runni
You can't take a snapshot if one is currently in progress. To check the status:
```
GET _snapshot/_status
GET /_snapshot/_status
```
{% include copy-curl.html %}
## Restore snapshots
@ -312,25 +286,28 @@ GET _snapshot/_status
The first step in restoring a snapshot is retrieving existing snapshots. To see all snapshot repositories:
```
GET _snapshot/_all
GET /_snapshot/_all
```
{% include copy-curl.html %}
To see all snapshots in a repository:
```
GET _snapshot/my-repository/_all
GET /_snapshot/my-repository/_all
```
{% include copy-curl.html %}
Then restore a snapshot:
```
POST _snapshot/my-repository/2/_restore
POST /_snapshot/my-repository/2/_restore
```
{% include copy-curl.html %}
Just like when taking a snapshot, you can add a request body to include or exclude certain indices or specify some other settings:
```json
POST _snapshot/my-repository/2/_restore
POST /_snapshot/my-repository/2/_restore
{
"indices": "opensearch-dashboards*,my-index*",
"ignore_unavailable": true,
@ -347,8 +324,9 @@ POST _snapshot/my-repository/2/_restore
]
}
```
{% include copy-curl.html %}
Request fields | Description
Request parameters | Description
:--- | :---
`indices` | The indices you want to restore. You can use `,` to create a list of indices, `*` to specify an index pattern, and `-` to exclude certain indices. Don't put spaces between items. Default is all indices.
`ignore_unavailable` | If an index from the `indices` list doesn't exist, whether to ignore it rather than fail the restore operation. Default is false.
@ -385,18 +363,20 @@ If you're using the security plugin, snapshots have some additional restrictions
If a snapshot contains global state, you must exclude it when performing the restore. If your snapshot also contains the `.opendistro_security` index, either exclude it or list all the other indices you want to include:
```json
POST _snapshot/my-repository/3/_restore
POST /_snapshot/my-repository/3/_restore
{
"indices": "-.opendistro_security",
"include_global_state": false
}
```
{% include copy-curl.html %}
The `.opendistro_security` index contains sensitive data, so we recommend excluding it when you take a snapshot. If you do need to restore the index from a snapshot, you must include an admin certificate in the request:
```bash
curl -k --cert ./kirk.pem --key ./kirk-key.pem -XPOST 'https://localhost:9200/_snapshot/my-repository/3/_restore?pretty'
```
{% include copy-curl.html %}
We strongly recommend against restoring `.opendistro_security` using an admin certificate because doing so can alter the security posture of the entire cluster. See [A word of caution]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin/#a-word-of-caution) for a recommended process to back up and restore your security plugin configuration.
{: .warning}