178 lines
6.5 KiB
Plaintext
178 lines
6.5 KiB
Plaintext
[role="xpack"]
|
|
[testenv="basic"]
|
|
[[getting-started-snapshot-lifecycle-management]]
|
|
=== Tutorial: Automate backups with {slm-init}
|
|
|
|
This tutorial demonstrates how to automate daily backups of {es} indices using an {slm-init} policy.
|
|
The policy takes <<modules-snapshots, snapshots>> of all indices in the cluster
|
|
and stores them in a local repository.
|
|
It also defines a retention policy and automatically deletes snapshots
|
|
when they are no longer needed.
|
|
|
|
To manage snapshots with {slm-init}, you:
|
|
|
|
. <<slm-gs-register-repository, Register a repository>>.
|
|
. <<slm-gs-create-policy, Create an {slm-init} policy>>.
|
|
|
|
To test the policy, you can manually trigger it to take an initial snapshot.
|
|
|
|
[discrete]
|
|
[[slm-gs-register-repository]]
|
|
==== Register a repository
|
|
|
|
To use {slm-init}, you must have a snapshot repository configured.
|
|
The repository can be local (shared filesystem) or remote (cloud storage).
|
|
Remote repositories can reside on S3, HDFS, Azure, Google Cloud Storage,
|
|
or any other platform supported by a {plugins}/repository.html[repository plugin].
|
|
Remote repositories are generally used for production deployments.
|
|
|
|
For this tutorial, you can register a local repository from
|
|
{kibana-ref}/snapshot-repositories.html[{kib} Management]
|
|
or use the put repository API:
|
|
|
|
[source,console]
|
|
-----------------------------------
|
|
PUT /_snapshot/my_repository
|
|
{
|
|
"type": "fs",
|
|
"settings": {
|
|
"location": "my_backup_location"
|
|
}
|
|
}
|
|
-----------------------------------
|
|
|
|
[discrete]
|
|
[[slm-gs-create-policy]]
|
|
==== Set up a snapshot policy
|
|
|
|
Once you have a repository in place,
|
|
you can define an {slm-init} policy to take snapshots automatically.
|
|
The policy defines when to take snapshots, which indices should be included,
|
|
and what to name the snapshots.
|
|
A policy can also specify a <<slm-retention,retention policy>> and
|
|
automatically delete snapshots when they are no longer needed.
|
|
|
|
TIP: Don't be afraid to configure a policy that takes frequent snapshots.
|
|
Snapshots are incremental and make efficient use of storage.
|
|
|
|
You can define and manage policies through {kib} Management or with the put policy API.
|
|
|
|
For example, you could define a `nightly-snapshots` policy
|
|
to back up all of your indices daily at 2:30AM UTC.
|
|
|
|
A put policy request defines the policy configuration in JSON:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /_slm/policy/nightly-snapshots
|
|
{
|
|
"schedule": "0 30 1 * * ?", <1>
|
|
"name": "<nightly-snap-{now/d}>", <2>
|
|
"repository": "my_repository", <3>
|
|
"config": { <4>
|
|
"indices": ["*"] <5>
|
|
},
|
|
"retention": { <6>
|
|
"expire_after": "30d", <7>
|
|
"min_count": 5, <8>
|
|
"max_count": 50 <9>
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
<1> When the snapshot should be taken in
|
|
<<schedule-cron,Cron syntax>>: daily at 2:30AM UTC
|
|
<2> How to name the snapshot: use
|
|
<<date-math-index-names,date math>> to include the current date in the snapshot name
|
|
<3> Where to store the snapshot
|
|
<4> The configuration to be used for the snapshot requests (see below)
|
|
<5> Which indices to include in the snapshot: all indices
|
|
<6> Optional retention policy: keep snapshots for 30 days,
|
|
retaining at least 5 and no more than 50 snapshots regardless of age
|
|
|
|
You can specify additional snapshot configuration options to customize how snapshots are taken.
|
|
For example, you could configure the policy to fail the snapshot
|
|
if one of the specified indices is missing.
|
|
For more information about snapshot options, see <<snapshots-take-snapshot,snapshot requests>>.
|
|
|
|
[discrete]
|
|
[[slm-gs-test-policy]]
|
|
==== Test the snapshot policy
|
|
|
|
A snapshot taken by {slm-init} is just like any other snapshot.
|
|
You can view information about snapshots in {kib} Management or
|
|
get info with the <<snapshots-monitor-snapshot-restore, snapshot APIs>>.
|
|
In addition, {slm-init} keeps track of policy successes and failures so you
|
|
have insight into how the policy is working. If the policy has executed at
|
|
least once, the <<slm-api-get-policy, get policy>> API returns additional metadata
|
|
that shows if the snapshot succeeded.
|
|
|
|
You can manually execute a snapshot policy to take a snapshot immediately.
|
|
This is useful for taking snapshots before making a configuration change,
|
|
upgrading, or to test a new policy.
|
|
Manually executing a policy does not affect its configured schedule.
|
|
|
|
For example, the following request manually triggers the `nightly-snapshots` policy:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
POST /_slm/policy/nightly-snapshots/_execute
|
|
--------------------------------------------------
|
|
// TEST[skip:we can't easily handle snapshots from docs tests]
|
|
|
|
|
|
After forcing the `nightly-snapshots` policy to run,
|
|
you can retrieve the policy to get success or failure information.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
GET /_slm/policy/nightly-snapshots?human
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
Only the most recent success and failure are returned,
|
|
but all policy executions are recorded in the `.slm-history*` indices.
|
|
The response also shows when the policy is scheduled to execute next.
|
|
|
|
NOTE: The response shows if the policy succeeded in _initiating_ a snapshot.
|
|
However, that does not guarantee that the snapshot completed successfully.
|
|
It is possible for the initiated snapshot to fail if, for example, the connection to a remote
|
|
repository is lost while copying files.
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"nightly-snapshots" : {
|
|
"version": 1,
|
|
"modified_date": "2019-04-23T01:30:00.000Z",
|
|
"modified_date_millis": 1556048137314,
|
|
"policy" : {
|
|
"schedule": "0 30 1 * * ?",
|
|
"name": "<nightly-snap-{now/d}>",
|
|
"repository": "my_repository",
|
|
"config": {
|
|
"indices": ["*"],
|
|
},
|
|
"retention": {
|
|
"expire_after": "30d",
|
|
"min_count": 5,
|
|
"max_count": 50
|
|
}
|
|
},
|
|
"last_success": {
|
|
"snapshot_name": "nightly-snap-2019.04.24-tmtnyjtrsxkhbrrdcgg18a", <1>
|
|
"time_string": "2019-04-24T16:43:49.316Z", <2>
|
|
"time": 1556124229316
|
|
} ,
|
|
"next_execution": "2019-04-24T01:30:00.000Z", <3>
|
|
"next_execution_millis": 1556048160000
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[skip:the presence of last_failure and last_success is asynchronous and will be present for users, but is untestable]
|
|
|
|
<1> The name of the last snapshot that was succesfully initiated by the policy
|
|
<2> When the snapshot was initiated
|
|
<3> When the policy will initiate the next snapshot
|
|
|