iSharkFly-Docs
/
opensearch-docs-cn
mirror of https://github.com/iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Snapshot management (#757) 

* Added snapshot management

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Snapshot management v2

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added snapshot management

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added user roles

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added SME comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Changed dashboards access from index management to snapshot management

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Editorial comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
This commit is contained in:
kolchfa-aws 2022-07-07 14:19:01 -04:00 committed by GitHub
parent 8e43230cc1
commit f225c77c34
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 678 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
_im-plugin/ism/policies.md
View File

@ -374,7 +374,7 @@ Parameter | Description | Type
### snapshot
Backup your cluster’s indexes and state. For more information about snapshots, see [Take and restore snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/).
Back up your cluster’s indexes and state. For more information about snapshots, see [Take and restore snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore).
The `snapshot` operation has the following parameters:

2
_monitoring-plugins/alerting/settings.md
View File

@ -16,7 +16,7 @@ Index | Purpose
:--- | :---
`.opendistro-alerting-alerts` | Stores ongoing alerts.
`.opendistro-alerting-alert-history-<date>` | Stores a history of completed alerts.
`.opendistro-alerting-config` | Stores monitors, triggers, and destinations. [Take a snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore) of this index to back up your alerting configuration.
`.opendistro-alerting-config` | Stores monitors, triggers, and destinations. [Take a snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore) of this index to back up your alerting configuration.
`.opendistro-alerting-alert-history-write` (alias) | Provides a consistent URI for the `.opendistro-alerting-alert-history-<date>` index.
All alerting indices are hidden by default. For a summary, make the following request:

27
_opensearch/snapshots/index.md Normal file
View File

@ -0,0 +1,27 @@
---
layout: default
title: Snapshots
nav_order: 65
has_children: true
redirect_from: /opensearch/snapshots/
has_toc: false
---
# Snapshots
Snapshots are backups of a cluster's indexes and state. State includes cluster settings, node information, index metadata (mappings, settings, templates, etc.), and shard allocation.
Snapshots have two main uses:
- **Recovering from failure**
For example, if cluster health goes red, you might restore the red indexes from a snapshot.
- **Migrating from one cluster to another**
For example, if you're moving from a proof-of-concept to a production cluster, you might take a snapshot of the former and restore it on the latter.
You can take and restore snapshots using the [snapshot API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore).
If you need to automate taking snapshots, you can use the [Snapshot Management]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-management) feature.

460
_opensearch/snapshots/sm-api.md Normal file
View File

@ -0,0 +1,460 @@
---
layout: default
title: Snapshot Management API
parent: Snapshots
nav_order: 30
has_children: false
---
# Snapshot Management API
Use the [Snapshot Management (SM)]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots) API to automate [taking snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots).
---
#### Table of contents
- TOC
{:toc}
---
## Create or update a policy
Introduced 2.1
{: .label .label-purple }
Creates or updates an SM policy.
#### Request
Create:
```json
POST _plugins/_sm/policies/<policy_name>
```
Update:
```json
PUT _plugins/_sm/policies/<policy_name>?if_seq_no=0&if_primary_term=1
```
You must provide the `seq_no` and `primary_term` parameters for an update request.
### Example
```json
POST _plugins/_sm/policies/daily-policy
{
"description": "Daily snapshot policy",
"creation": {
"schedule": {
"cron": {
"expression": "0 8 * * *",
"timezone": "UTC"
}
},
"time_limit": "1h"
},
"deletion": {
"schedule": {
"cron": {
"expression": "0 1 * * *",
"timezone": "America/Los_Angeles"
}
},
"condition": {
"max_age": "7d",
"max_count": 21,
"min_count": 7
},
"time_limit": "1h"
},
"snapshot_config": {
"date_format": "yyyy-MM-dd-HH:mm",
"timezone": "America/Los_Angeles",
"indices": "*",
"repository": "s3-repo",
"ignore_unavailable": "true",
"include_global_state": "false",
"partial": "true",
"metadata": {
"any_key": "any_value"
}
},
"notification": {
"channel": {
"id": "NC3OpoEBzEoHMX183R3f"
},
"conditions": {
"creation": true,
"deletion": false,
"failure": false,
"time_limit_exceeded": false
}
}
}
```
### Response
```json
{
"_id" : "daily-policy-sm-policy",
"_version" : 5,
"_seq_no" : 54983,
"_primary_term" : 21,
"sm_policy" : {
"name" : "daily-policy",
"description" : "Daily snapshot policy",
"schema_version" : 15,
"creation" : {
"schedule" : {
"cron" : {
"expression" : "0 8 * * *",
"timezone" : "UTC"
}
},
"time_limit" : "1h"
},
"deletion" : {
"schedule" : {
"cron" : {
"expression" : "0 1 * * *",
"timezone" : "America/Los_Angeles"
}
},
"condition" : {
"max_age" : "7d",
"min_count" : 7,
"max_count" : 21
},
"time_limit" : "1h"
},
"snapshot_config" : {
"indices" : "*",
"metadata" : {
"any_key" : "any_value"
},
"ignore_unavailable" : "true",
"timezone" : "America/Los_Angeles",
"include_global_state" : "false",
"date_format" : "yyyy-MM-dd-HH:mm",
"repository" : "s3-repo",
"partial" : "true"
},
"schedule" : {
"interval" : {
"start_time" : 1656425122909,
"period" : 1,
"unit" : "Minutes"
}
},
"enabled" : true,
"last_updated_time" : 1656425122909,
"enabled_time" : 1656425122909,
"notification" : {
"channel" : {
"id" : "NC3OpoEBzEoHMX183R3f"
},
"conditions" : {
"creation" : true,
"deletion" : false,
"failure" : false,
"time_limit_exceeded" : false
}
}
}
}
```
### Parameters
You can specify the following parameters to create/update an SM policy.
Parameter | Type | Description
:--- | :--- |:--- |:--- |
`description` | String | The description of the SM policy. Optional.
`enabled` | Boolean | Should this SM policy be enabled at creation? Optional.
`snapshot_config` | Object | The configuration options for snapshot creation. Required.
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`date_format` | String | Snapshot names have the format `<policy_name>-<date>-<random number>`. `date_format` specifies the format for the date in the snapshot name. Supports all date formats supported by OpenSearch. Optional. Default is "yyyy-MM-dd'T'HH:mm:ss".
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`date_format_timezone` | String | Snapshot names have the format `<policy_name>-<date>-<random number>`. `date_format_timezone` specifies the time zone for the date in the snapshot name. Optional. Default is UTC.
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`indices` | String | The names of the indexes in the snapshot. Multiple index names are separated by `,`. Supports wildcards (`*`). Optional. Default is `*` (all indexes).
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`repository` | String | The repository in which to store snapshots. Required.
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`ignore_unavailable` | Boolean | Do you want to ignore unavailable indexes? Optional. Default is `false`.
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`include_global_state` | Boolean | Do you want to include cluster state? Optional. Default is `true` because of [Security plugin considerations]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore/#security-plugin-considerations).
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`partial` | Boolean | Do you want to allow partial snapshots? Optional. Default is `false`.
`snapshot_config`<br>&nbsp;&nbsp;&nbsp;&nbsp;`metadata` | Object | Metadata in the form of key/value pairs. Optional.
`creation` | Object | Configuration for snapshot creation. Required.
`creation`<br>&nbsp;&nbsp;&nbsp;&nbsp;`schedule` | String | The cron schedule used to create snapshots. Required.
`creation`<br>&nbsp;&nbsp;&nbsp;&nbsp;`time_limit` | String | Sets the maximum time to wait for snapshot creation to finish. If time_limit is longer than the scheduled time interval for taking snapshots, no scheduled snapshots are taken until time_limit elapses. For example, if time_limit is set to 35 minutes and snapshots are taken every 30 minutes starting at midnight, the snapshots at 00:00 and 01:00 are taken, but the snapshot at 00:30 is skipped. Optional.
`deletion` | Object | Configuration for snapshot deletion. Optional. Default is to retain all snapshots.
`deletion`<br>&nbsp;&nbsp;&nbsp;&nbsp;`schedule` | String | The cron schedule used to delete snapshots. Optional. Default is to use `creation.schedule`, which is required.
`deletion`<br>&nbsp;&nbsp;&nbsp;&nbsp;`time_limit` | String | Sets the maximum time to wait for snapshot deletion to finish. Optional.
`deletion`<br>&nbsp;&nbsp;&nbsp;&nbsp;`delete_condition` | Object | Conditions for snapshot deletion. Optional.
`deletion`<br>&nbsp;&nbsp;&nbsp;&nbsp;`delete_condition`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`max_count` | Integer | The maximum number of snapshots to be retained. Optional.
`deletion`<br>&nbsp;&nbsp;&nbsp;&nbsp;`delete_condition`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`max_age` | String | The maximum time a snapshot is retained. Optional.
`deletion`<br>&nbsp;&nbsp;&nbsp;&nbsp;`delete_condition`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`min_count` | Integer | The minimum number of snapshots to be retained. Optional. Default is one.
`notification` | Object | Defines notifications for SM events. Optional.
`notification`<br>&nbsp;&nbsp;&nbsp;&nbsp;`channel` | Object | Defines a channel for notifications. You must [create and configure a notification channel]({{site.url}}{{site.baseurl}}/notifications-plugin/api) before setting up SM notifications. Required.
`notification`<br>&nbsp;&nbsp;&nbsp;&nbsp;`channel`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`id` | String | The channel ID of the channel used for notifications. To get the channel IDs of all created channels, use `GET _plugins/_notifications/configs`. Required.
`notification`<br>&nbsp;&nbsp;&nbsp;&nbsp;`conditions` | Object | SM events you want to be notified about. Set the ones you are interested in to `true`.
`notification`<br>&nbsp;&nbsp;&nbsp;&nbsp;`conditions`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`creation` | Boolean | Do you want notifications about snapshot creation? Optional. Default is `true`.
`notification`<br>&nbsp;&nbsp;&nbsp;&nbsp;`conditions`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`deletion` | Boolean | Do you want notifications about snapshot deletion? Optional. Default is `false`.
`notification`<br>&nbsp;&nbsp;&nbsp;&nbsp;`conditions`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`failure` | Boolean | Do you want notifications about creation or deletion failure? Optional. Default is `false`.
`notification`<br>&nbsp;&nbsp;&nbsp;&nbsp;`conditions`<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;`time_limit_exceeded` | Boolean | Do you want notifications when snapshot operations take longer than time_limit? Optional. Default is `false`.
## Get policies
Introduced 2.1
{: .label .label-purple }
Gets SM policies.
#### Request
Get all SM policies:
```json
GET _plugins/_sm/policies
```
You can use a [query string]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/#query-string) and specify pagination, the field to be sorted by, and sort order:
```json
GET _plugins/_sm/policies?from=0&size=20&sortField=sm_policy.name&sortOrder=desc&queryString=*
```
Get a specific SM policy:
```
GET _plugins/_sm/policies/<policy_name>
```
### Example
```json
GET _plugins/_sm/policies/daily-policy
```
### Response
```json
{
"_id" : "daily-policy-sm-policy",
"_version" : 6,
"_seq_no" : 44696,
"_primary_term" : 19,
"sm_policy" : {
"name" : "daily-policy",
"description" : "Daily snapshot policy",
"schema_version" : 15,
"creation" : {
"schedule" : {
"cron" : {
"expression" : "0 8 * * *",
"timezone" : "UTC"
}
},
"time_limit" : "1h"
},
"deletion" : {
"schedule" : {
"cron" : {
"expression" : "0 1 * * *",
"timezone" : "America/Los_Angeles"
}
},
"condition" : {
"max_age" : "7d",
"min_count" : 7,
"max_count" : 21
},
"time_limit" : "1h"
},
"snapshot_config" : {
"metadata" : {
"any_key" : "any_value"
},
"ignore_unavailable" : "true",
"include_global_state" : "false",
"date_format" : "yyyy-MM-dd-HH:mm",
"repository" : "s3-repo",
"partial" : "true"
},
"schedule" : {
"interval" : {
"start_time" : 1656341042874,
"period" : 1,
"unit" : "Minutes"
}
},
"enabled" : true,
"last_updated_time" : 1656341042874,
"enabled_time" : 1656341042874
}
}
```
## Explain
Introduced 2.1
{: .label .label-purple }
Provides the enabled/disabled status and the metadata for all policies specified. Multiple policy names are separated with `,`. You can also specify desired policies with a wildcard pattern.
<img src="{{site.url}}{{site.baseurl}}/images/sm-state-machine.png" alt="SM State Machine" width="150" style="float: left; margin-right: 15px;"/>
SM uses a state machine for snapshot creation and deletion. The image on the left shows one execution period of the creation workflow, from the CREATION_START state to the CREATION_FINISHED state. Deletion workflow follows the same pattern as creation workflow.
The creation workflow starts in the CREATION_START state and continuously checks if the conditions in the creation cron schedule are met. After the conditions are met, the creation workflow switches to the CREATION_CONDITION_MET state and continues to the CREATING state. The CREATING state calls the create snapshot API asynchronously and then waits for snapshot creation to end in the CREATION_FINISHED state. Once snapshot creation ends, the creation workflow goes back to the CREATION_START state, and the cycle continues. The `current_state` field of `metadata.creation` and `metadata.deletion` returns the current state of the state machine.
#### Request
```json
GET _plugins/_sm/policies/<policy_names>/_explain
```
### Example
```json
GET _plugins/_sm/policies/daily*/_explain
```
### Response
```json
{
"policies" : [
{
"name" : "daily-policy",
"creation" : {
"current_state" : "CREATION_START",
"trigger" : {
"time" : 1656403200000
}
},
"deletion" : {
"current_state" : "DELETION_START",
"trigger" : {
"time" : 1656403200000
}
},
"policy_seq_no" : 44696,
"policy_primary_term" : 19,
"enabled" : true
}
]
}
```
The following table lists all fields for each policy in the response.
Field | Description
:--- |:---
`name` | The name of the SM policy.
`creation` | Information about the latest creation operation. See subfields below.
`deletion` | Information about the latest deletion operation. See subfields below.
`policy_seq_no` <br> `policy_primary_term` | The version of the SM policy.
`enabled` | Is the policy running?
The following table lists all fields in the `creation` and `deletion` objects of each policy.
Field | Description
:--- |:---
`current_state` | The current state of the state machine that runs snapshot creation/deletion as described above.
`trigger.time` | The next creation/deletion execution time in milliseconds since the epoch.
`latest_execution` | Describes the latest creation/deletion execution.
`latest_execution.status` | The execution status of the latest creation/deletion. Possible values are:<br> `IN_PROGRESS`: Snapshot creation/deletion has started. <br> `SUCCESS`: Snapshot creation/deletion has finished successfully. <br> `RETRYING`: The creation/deletion attempt has failed. It will be retried three times. <br> `FAILED`: The creation/deletion attempt failed after three retries. End the current execution period and go to the next execution period. <br> `TIME_LIMIT_EXCEEDED`: The creation/deletion time exceeded the time_limit set in the policy. End the current execution period and go to the next execution period.
`latest_execution.start_time` | The start time of the latest execution in milliseconds since the epoch.
`latest_execution.end_time` | The end time of the latest execution in milliseconds since the epoch.
`latest_execution.info.message` | A user-friendly message describing the status of the latest execution.
`latest_execution.info.cause` | Contains the failure reason if the latest execution fails.
`retry.count` | The number of remaining execution retry attempts.
## Start a policy
Introduced 2.1
{: .label .label-purple }
Starts the policy by setting its `enabled` flag to `true`.
#### Request
```json
POST _plugins/_sm/policies/<policy_name>/_start
```
### Example
```json
POST _plugins/_sm/policies/daily-policy/_start
```
### Response
```json
{
"acknowledged" : true
}
```
## Stop a policy
Introduced 2.1
{: .label .label-purple }
Sets the `enabled` flag to `false` for an SM policy. The policy will not run until you [start](#start-a-policy) it.
#### Request
```json
POST _plugins/_sm/policies/<policy_name>/_stop
```
### Example
```json
POST _plugins/_sm/policies/daily-policy/_stop
```
### Response
```json
{
"acknowledged" : true
}
```
## Delete a policy
Introduced 2.1
{: .label .label-purple }
Deletes the specified SM policy.
#### Request
```json
DELETE _plugins/_sm/policies/<policy_name>
```
### Example
```json
DELETE _plugins/_sm/policies/daily-policy
```
### Response
```json
{
"_index" : ".opendistro-ism-config",
"_id" : "daily-policy-sm-policy",
"_version" : 8,
"result" : "deleted",
"forced_refresh" : true,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"_seq_no" : 45366,
"_primary_term" : 20
}
```

97
_opensearch/snapshots/sm-dashboards.md Normal file
View File

@ -0,0 +1,97 @@
---
layout: default
title: Snapshot Management in OpenSearch Dashboards
parent: Snapshots
nav_order: 35
has_children: false
---
# Using Snapshot Management in OpenSearch Dashboards
You can set up Snapshot Management (SM) in OpenSearch Dashboards.
## Create a repository
Before you create an SM policy, you need to set up a repository for snapshots.
1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**.
1. In the left panel, under **Snapshot Management**, select **Repositories**.
1. Select the **Create Repository** button.
1. Enter the repository name, type, and location.
1. (Optional) Select **Advanced Settings** and enter additional settings for this repository as a JSON object. Example:
```json
{
"chunk_size": null,
"compress": false,
"max_restore_bytes_per_sec": "40m",
"max_snapshot_bytes_per_sec": "40m",
"readonly": false
}
```
1. Select the **Add** button.
## Create an SM policy
Create an SM policy to set up automatic snapshots. An SM policy defines an automated snapshot creation schedule and an optional automated deletion schedule.
1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**.
1. In the left panel, under **Snapshot Management**, select **Snapshot Policies**.
1. Select the **Create Policy** button.
1. In the **Policy settings** section:
1. Enter the policy name.
1. (Optional) Enter the policy description.
1. In the **Source and destination** section:
1. Select or enter source indexes either as a list or as an index pattern.
1. Select a repository for snapshots. To [create a new repository](#create-a-repository), select the **Create** button.
1. In the **Snapshot schedule** section:
1. Select the desired snapshot frequency or enter a custom cron expression for snapshot frequency.
1. Select the start time and time zone.
1. In the **Retention period** section:
1. Choose to retain all snapshots or specify retention conditions (the maximum age of retained snapshots).
1. (Optional) In **Additional settings**, select the minimum and maximum number of retained snapshots, deletion frequency, and deletion start time.
1. In the **Notifications** section, select the snapshot activities you want to be notified about.
1. (Optional) In the **Advanced settings** section, select the desired options:
- **Include cluster state in snapshots**
- **Ignore unavailable indices**
- **Allow partial snapshots**
1. Select the **Create** button.
## View, edit, or delete an SM policy
You can view, edit, or delete an SM policy on the policy details page.
1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**.
1. In the left panel, under **Snapshot Management**, select **Snapshot Policies**.
1. Click on the **Policy name** of the policy you want to view, edit, or delete. <br>
The policy settings, snapshot schedule, snapshot retention period, notifications, and last creation and deletion are displayed in the policy details page. <br> If a snapshot creation or deletion fails, you can view information about the failure in the **Last Creation/Deletion** section. To view the failure message, click on the **cause** in the **Info** column.
1. To edit or delete the SM policy, select the **Edit** or **Delete** button.
## Enable, disable, or delete SM policies
1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**.
1. In the left panel, under **Snapshot Management**, select **Snapshot Policies**.
1. Select one or more policies in the list.
1. To enable or disable selected SM policies, select the **Enable** or **Disable** button. To delete selected SM policies, in the **Actions** list, select the **Delete** option.
## View snapshots
1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**.
1. In the left panel, under **Snapshot Management**, select **Snapshots**.
All automatically or manually taken snapshots appear in the list.
1. To view a snapshot, click on its **Name**.
## Take a snapshot
Use the steps below to take a snapshot manually. If you need to restore a snapshot, use the [restore snapshot API operation]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#restore-snapshots).
1. On the top menu bar, go to **OpenSearch Plugins > Snapshot Management**.
1. In the left panel, under **Snapshot Management**, select **Snapshots**.
1. Select the **Take snapshot** button.
1. Enter the snapshot name.
1. Select or enter source indexes either as a list or as an index pattern.
1. Select a repository for the snapshot.
1. (Optional) In the **Advanced options** section, select the desired options:
- **Include cluster state in snapshots**
- **Ignore unavailable indices**
- **Allow partial snapshots**
1. Select the **Add** button.

78
_opensearch/snapshots/snapshot-management.md Normal file
View File

@ -0,0 +1,78 @@
---
layout: default
title: Snapshot Management
parent: Snapshots
nav_order: 20
has_children: false
---
# Snapshot Management
Snapshot Management (SM) lets you automate [taking snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots). To use this feature, you need to install the [Index Management (IM) Plugin]({{site.url}}{{site.baseurl}}/im-plugin). Snapshots store only incremental changes since the last snapshot. Thus, while taking an initial snapshot may be a heavy operation, subsequent snapshots have minimal overhead. To set up automatic snapshots, you have to create an SM policy with a desired SM schedule and configuration.
When you create an SM policy, its document ID is given the name `<policy_name>-sm-policy`. Because of this, SM policies have to obey the following rules:
- SM policies must have unique names.
- You cannot update the policy name after its creation.
SM-created snapshots have names in the format `<policy_name>-<date>-<random number>`. Two snapshots created by different policies at the same time always have different names because of the `<policy_name>` prefix. To avoid name collisions within the same policy, each snapshot's name contains a random string suffix.
Each policy has associated metadata that stores the policy status. Snapshot Management saves SM policies and metadata in the system index and reads them from the system index. Thus, Snapshot Management depends on the OpenSearch cluster's indexing and searching functions. The policy's metadata keeps information about the latest creation and deletion only. The metadata is read before running every scheduled job so that SM can continue execution from the previous job's state. You can view the metadata using the [explain API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#explain).
An SM schedule is a custom [cron]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron) expression. It consists of two parts: a creation schedule and a deletion schedule. You must set up a creation schedule that specifies the frequency and timing of snapshot creation. Optionally, you can set up a separate schedule for deleting snapshots.
An SM configuration includes the indexes and repository for the snapshots and supports all parameters you can define when [creating a snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#take-snapshots) using the API. Additionally, you can specify the format and time zone for the date used in the snapshot's name.
## Performance
One snapshot can contain as many indexes as there are in the cluster. We expect at most dozens of SM policies in one cluster, but a snapshot repository can safely scale to thousands of snapshots. However, to manage its metadata, a large repository requires more memory on the cluster manager node.
Snapshot Management depends on the Job Scheduler plugin to schedule a job that is run periodically. Each SM policy corresponds to one SM-scheduled job. The scheduled job is lightweight, so the burden of SM depends on the snapshot creation frequency and the burden of running the snapshot operation itself.
## Concurrency
An SM policy does not support concurrent snapshot operations, since too many such operations may degrade the cluster. Snapshot operations (creation or deletion) are performed asynchronously. SM does not start a new operation until the previous asynchronous operation finishes.
We don't recommend creating several SM policies with the same schedule and overlapping indexes in one cluster because it leads to concurrent snapshot creation on the same indexes and hinders performance.
{: .warning }
We don't recommend setting up the same repository for multiple SM policies with same schedule in different clusters, since it may cause a sudden spike of burden in this repository.
{: .warning }
## Failure management
If a snapshot operation fails, it is retried a maximum of three times. The failure message is saved in `metadata.latest_execution` and is overwritten when a subsequent snapshot operation starts. You can view the failure message using the [explain API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#explain). When using OpenSearch Dashboards, you can view the failure message on the [policy details page]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-dashboards#view-edit-or-delete-an-sm-policy). Possible reasons for failure include red index status and shard reallocation.
## Security
The Security plugin has two built-in roles for Snapshot Management actions: `snapshot_management_full_access` and `snapshot_management_read_access`. For descriptions of each, see [Predefined roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#predefined-roles).
The following table lists the required permissions for each Snapshot Management API.
Function | API | Permission
:--- | :--- | :---
Get policy | GET _plugins/_sm/policies<br>GET _plugins/_sm/policies/`policy_name` | cluster:admin/opensearch/snapshot_management/policy/get<br>cluster:admin/opensearch/snapshot_management/policy/search
Create/update policy | POST _plugins/_sm/policies/`policy_name`<br> PUT _plugins/_sm/policies/`policy_name`?if_seq_no=1&if_primary_term=1 | cluster:admin/opensearch/snapshot_management/policy/write
Delete policy | DELETE _plugins/_sm/policies/`policy_name` | cluster:admin/opensearch/snapshot_management/policy/delete
Explain | GET _plugins/_sm/policies/`policy_names`/_explain | cluster:admin/opensearch/snapshot_management/policy/explain
Start | POST _plugins/_sm/policies/`policy_name`/_start | cluster:admin/opensearch/snapshot_management/policy/start
Stop| POST _plugins/_sm/policies/`policy_name`/_stop | cluster:admin/opensearch/snapshot_management/policy/stop
## API
The following table lists all [Snapshot Management API]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api) functions.
Function | API | Description
:--- | :--- | :---
[Create policy]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#create-or-update-a-policy) | POST _plugins/_sm/policies/`policy_name` | Creates an SM policy.
[Update policy]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#create-or-update-a-policy) | PUT _plugins/_sm/policies/`policy_name`?if_seq_no=`sequence_number`&if_primary_term=`primary_term` | Modifies the `policy_name` policy.
[Get all policies]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#get-policies) | GET _plugins/_sm/policies | Returns all SM policies.
[Get the policy `policy_name`]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#get-policies) | GET _plugins/_sm/policies/`policy_name` | Returns the `policy_name` SM policy.
[Delete policy]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#delete-a-policy) | DELETE _plugins/_sm/policies/`policy_name` | Deletes the `policy_name` policy.
[Explain]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#explain) | GET _plugins/_sm/policies/`policy_names`/_explain | Provides the enabled/disabled status and the metadata for all policies specified by `policy_names`.
[Start policy]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#start-a-policy) | POST _plugins/_sm/policies/`policy_name`/_start | Starts the `policy_name` policy.
[Stop policy]({{site.url}}{{site.baseurl}}/opensearch/snapshots/sm-api#stop-a-policy)| POST _plugins/_sm/policies/`policy_name`/_stop | Stops the `policy_name` policy.

30
_opensearch/snapshot-restore.md → _opensearch/snapshots/snapshot-restore.md
View File

@ -1,23 +1,21 @@
---
layout: default
title: Take and restore snapshots
nav_order: 65
parent: Snapshots
nav_order: 10
has_children: false
---
# Take and restore snapshots
Snapshots are backups of a cluster's indices and state. State includes cluster settings, node information, index metadata (mappings, settings, templates, etc.), and shard allocation.
Snapshots aren't instantaneous. They take time to complete and do not represent perfect point-in-time views of the cluster. While a snapshot is in progress, you can still index documents and send other requests to the cluster, but new documents and updates to existing documents generally aren't included in the snapshot. The snapshot includes primary shards as they existed when OpenSearch initiated the snapshot. Depending on the size of your snapshot thread pool, different shards might be included in the snapshot at slightly different times.
Snapshots have two main uses:
OpenSearch snapshots are incremental, meaning that they only store data that has changed since the last successful snapshot. The difference in disk usage between frequent and infrequent snapshots is often minimal.
- **Recovering from failure**
For example, if cluster health goes red, you might restore the red indices from a snapshot.
- **Migrating from one cluster to another**
For example, if you're moving from a proof-of-concept to a production cluster, you might take a snapshot of the former and restore it on the latter.
In other words, taking hourly snapshots for a week (for a total of 168 snapshots) might not use much more disk space than taking a single snapshot at the end of the week. Also, the more frequently you take snapshots, the less time they take to complete. Some OpenSearch users take snapshots as often as every 30 minutes.
If you need to delete a snapshot, be sure to use the OpenSearch API rather than navigating to the storage location and purging files. Incremental snapshots from a cluster often share a lot of the same data; when you use the API, OpenSearch only removes data that no other snapshot is using.
{: .tip }
---
@ -28,18 +26,6 @@ Snapshots have two main uses:
---
## About snapshots
Snapshots aren't instantaneous. They take time to complete and do not represent perfect point-in-time views of the cluster. While a snapshot is in progress, you can still index documents and make other requests to the cluster, but new documents and updates to existing documents generally aren't included in the snapshot. The snapshot includes primary shards as they existed when OpenSearch initiated the snapshot. Depending on the size of your snapshot thread pool, different shards might be included in the snapshot at slightly different times.
OpenSearch snapshots are incremental, meaning that they only store data that has changed since the last successful snapshot. The difference in disk usage between frequent and infrequent snapshots is often minimal.
In other words, taking hourly snapshots for a week (for a total of 168 snapshots) might not use much more disk space than taking a single snapshot at the end of the week. Also, the more frequently you take snapshots, the less time they take to complete. Some OpenSearch users take snapshots as often as every half hour.
If you need to delete a snapshot, be sure to use the OpenSearch API rather than navigating to the storage location and purging files. Incremental snapshots from a cluster often share a lot of the same data; when you use the API, OpenSearch only removes data that no other snapshot is using.
{: .tip }
## Register repository
Before you can take a snapshot, you have to "register" a snapshot repository. A snapshot repository is just a storage location: a shared file system, Amazon S3, Hadoop Distributed File System (HDFS), Azure Storage, etc.

2
_security-plugin/access-control/multi-tenancy.md
View File

@ -165,4 +165,4 @@ The open source version of OpenSearch Dashboards saves all objects to a single i
The security plugin scrubs these index names of special characters, so they might not be a perfect match of tenant names and usernames.
{: .tip }
To back up your OpenSearch Dashboards data, [take a snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/) of all tenant indices using an index pattern such as `.kibana*`.
To back up your OpenSearch Dashboards data, [take a snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore/) of all tenant indexes using an index pattern such as `.kibana*`.

5
_security-plugin/access-control/users-roles.md
View File

@ -124,9 +124,10 @@ Role | Description
`reports_instances_read_access` | Grants permissions to generate on-demand reports and download existing reports, but not to view or create report definitions.
`reports_full_access` | Grants full permissions to reports.
`asynchronous_search_full_access` | Grants full permissions to all asynchronous search actions.
`asynchronous_search_read_access` | Grants permissions to view asynchronous searches, but not to submit, modify, or delete async searches.
`asynchronous_search_read_access` | Grants permissions to view asynchronous searches but not to submit, modify, or delete async searches.
`index_management_full_access` | Grants full permissions to all index management actions, including ISM, transforms, and rollups.
`snapshot_management_full_access` | Grants full permissions to all snapshot management actions.
`snapshot_management_read_access` | Grants permissions to view policies but not create, modify, start, stop, or delete them.
For more detailed summaries of the permissions for each role, reference their action groups against the descriptions in [Default action groups]({{site.url}}{{site.baseurl}}/security-plugin/access-control/default-action-groups/).

2
_upgrade-to/index.md
View File

@ -16,6 +16,6 @@ Three approaches exist:
- Perform a [rolling upgrade or cluster restart upgrade]({{site.url}}{{site.baseurl}}/upgrade-to/upgrade-to/) on your existing nodes.
- Replace existing Elasticsearch OSS nodes with new OpenSearch nodes. Node replacement is most popular when upgrading [Docker clusters]({{site.url}}{{site.baseurl}}/upgrade-to/docker-upgrade-to/).
Regardless of your approach, to safeguard against data loss, we recommend that you take a [snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/) of all indices prior to any migration.
Regardless of your approach, to safeguard against data loss, we recommend that you take a [snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore) of all indexes prior to any migration.
If your existing clients include a version check, such as recent versions of Logstash OSS and Filebeat OSS, [check compatibility]({{site.url}}{{site.baseurl}}/clients/agents-and-ingestion-tools/index/) before upgrading.

2
_upgrade-to/snapshot-migrate.md
View File

@ -6,6 +6,6 @@ nav_order: 5
# Use snapshots to migrate data
One popular approach is to take a [snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/) of your Elasticsearch OSS 6.x or 7.x indices, [create an OpenSearch cluster]({{site.url}}{{site.baseurl}}/opensearch/install/), restore the snapshot on the new cluster, and point your clients to the new host.
One popular approach is to take a [snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore) of your Elasticsearch OSS 6.x or 7.x indexes, [create an OpenSearch cluster]({{site.url}}{{site.baseurl}}/opensearch/install/), restore the snapshot on the new cluster, and point your clients to the new host.
The snapshot approach can mean running two clusters in parallel, but lets you validate that the OpenSearch cluster is working in a way that meets your needs prior to modifying the Elasticsearch OSS cluster.

BIN
images/SM-state-machine.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB