2014-02-18 17:28:49 -05:00
|
|
|
[[indices-recovery]]
|
2019-10-01 17:07:28 -04:00
|
|
|
=== Index recovery API
|
|
|
|
++++
|
|
|
|
<titleabbrev>Index recovery</titleabbrev>
|
|
|
|
++++
|
2014-02-18 17:28:49 -05:00
|
|
|
|
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
Returns information about ongoing and completed shard recoveries.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
GET /twitter/_recovery
|
|
|
|
----
|
|
|
|
// TEST[setup:twitter]
|
|
|
|
|
|
|
|
|
|
|
|
[[index-recovery-api-request]]
|
|
|
|
==== {api-request-title}
|
|
|
|
|
|
|
|
`GET /<index>/_recovery`
|
|
|
|
|
|
|
|
`GET /_recovery`
|
|
|
|
|
|
|
|
|
|
|
|
[[index-recovery-api-desc]]
|
|
|
|
==== {api-description-title}
|
|
|
|
|
|
|
|
Use the index recovery API
|
|
|
|
to get information about ongoing and completed shard recoveries.
|
|
|
|
|
|
|
|
// tag::shard-recovery-desc[]
|
|
|
|
Shard recovery is the process
|
|
|
|
of syncing a replica shard from a primary shard.
|
|
|
|
Upon completion,
|
|
|
|
the replica shard is available for search.
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
include::{es-repo-dir}/glossary.asciidoc[tag=recovery-triggers]
|
2019-10-03 16:43:26 -04:00
|
|
|
|
|
|
|
// end::shard-recovery-desc[]
|
|
|
|
|
|
|
|
[[index-recovery-api-path-params]]
|
|
|
|
==== {api-path-parms-title}
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index]
|
2019-10-03 16:43:26 -04:00
|
|
|
+
|
|
|
|
Use a value of `_all` to retrieve information for all indices in the cluster.
|
|
|
|
|
|
|
|
|
|
|
|
[[index-recovery-api-query-params]]
|
|
|
|
==== {api-query-parms-title}
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=active-only]
|
2019-10-03 16:43:26 -04:00
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=detailed]
|
2019-10-03 16:43:26 -04:00
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-query-parm]
|
2019-10-03 16:43:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
[[index-recovery-api-response-body]]
|
|
|
|
==== {api-response-body-title}
|
|
|
|
|
|
|
|
`id`::
|
|
|
|
(Integer)
|
|
|
|
ID of the shard.
|
|
|
|
|
|
|
|
`type`::
|
|
|
|
+
|
|
|
|
--
|
|
|
|
(String)
|
|
|
|
Recovery type.
|
|
|
|
Returned values include:
|
|
|
|
|
|
|
|
`STORE`::
|
|
|
|
The recovery is related to
|
|
|
|
a node startup or failure.
|
|
|
|
This type of recovery is called a local store recovery.
|
|
|
|
|
|
|
|
`SNAPSHOT`::
|
|
|
|
The recovery is related to
|
2020-01-09 19:55:46 -05:00
|
|
|
a <<snapshots-restore-snapshot,snapshot restoration>>.
|
2019-10-03 16:43:26 -04:00
|
|
|
|
|
|
|
`REPLICA`::
|
|
|
|
The recovery is related to
|
|
|
|
a <<glossary-replica-shard,primary shard replication>>.
|
|
|
|
|
|
|
|
`RELOCATING`::
|
|
|
|
The recovery is related to
|
|
|
|
the relocation of a shard
|
|
|
|
to a different node in the same cluster.
|
|
|
|
--
|
|
|
|
|
|
|
|
`STAGE`::
|
|
|
|
+
|
|
|
|
--
|
|
|
|
(String)
|
|
|
|
Recovery stage.
|
|
|
|
Returned values include:
|
|
|
|
|
|
|
|
`DONE`::
|
|
|
|
Complete.
|
|
|
|
|
|
|
|
`FINALIZE`::
|
|
|
|
Cleanup.
|
|
|
|
|
|
|
|
`INDEX`::
|
|
|
|
Reading index metadata and copying bytes from source to destination.
|
|
|
|
|
|
|
|
`INIT`::
|
|
|
|
Recovery has not started.
|
|
|
|
|
|
|
|
`START`::
|
|
|
|
Starting the recovery process; opening the index for use.
|
|
|
|
|
|
|
|
`TRANSLOG`::
|
|
|
|
Replaying transaction log .
|
|
|
|
--
|
|
|
|
|
|
|
|
`primary`::
|
|
|
|
(Boolean)
|
|
|
|
If `true`,
|
|
|
|
the shard is a primary shard.
|
|
|
|
|
|
|
|
`start_time`::
|
|
|
|
(String)
|
|
|
|
Timestamp of recovery start.
|
|
|
|
|
|
|
|
`stop_time`::
|
|
|
|
(String)
|
|
|
|
Timestamp of recovery finish.
|
|
|
|
|
|
|
|
`total_time_in_millis`::
|
|
|
|
(String)
|
|
|
|
Total time to recover shard in milliseconds.
|
|
|
|
|
|
|
|
`source`::
|
|
|
|
+
|
|
|
|
--
|
|
|
|
(Object)
|
|
|
|
Recovery source.
|
|
|
|
This can include:
|
|
|
|
|
|
|
|
* A repository description if recovery is from a snapshot
|
|
|
|
* A description of source node
|
|
|
|
--
|
|
|
|
|
|
|
|
`target`::
|
|
|
|
(Object)
|
|
|
|
Destination node.
|
|
|
|
|
|
|
|
`index`::
|
|
|
|
(Object)
|
|
|
|
Statistics about physical index recovery.
|
|
|
|
|
|
|
|
`translog`::
|
|
|
|
(Object)
|
|
|
|
Statistics about translog recovery.
|
|
|
|
|
|
|
|
`start`::
|
|
|
|
(Object)
|
|
|
|
Statistics about time to open and start the index.
|
|
|
|
|
|
|
|
|
|
|
|
[[index-recovery-api-example]]
|
|
|
|
==== {api-examples-title}
|
|
|
|
|
|
|
|
|
|
|
|
[[index-recovery-api-multi-ex]]
|
|
|
|
===== Get recovery information for several indices
|
2014-02-18 17:28:49 -05:00
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
[source,console]
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
2016-09-01 13:05:22 -04:00
|
|
|
GET index1,index2/_recovery?human
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
2016-09-01 13:05:22 -04:00
|
|
|
// TEST[s/^/PUT index1\nPUT index2\n/]
|
2014-02-18 17:28:49 -05:00
|
|
|
|
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
[[index-recovery-api-all-ex]]
|
|
|
|
===== Get segment information for all indices
|
2017-08-28 05:23:09 -04:00
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
//////////////////////////
|
2017-08-28 05:23:09 -04:00
|
|
|
Here we create a repository and snapshot index1 in
|
|
|
|
order to restore it right after and prints out the
|
2019-10-01 17:07:28 -04:00
|
|
|
index recovery result.
|
2017-08-28 05:23:09 -04:00
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
[source,console]
|
2017-08-28 05:23:09 -04:00
|
|
|
--------------------------------------------------
|
|
|
|
# create the index
|
2019-01-18 03:34:11 -05:00
|
|
|
PUT index1
|
2017-08-28 05:23:09 -04:00
|
|
|
{"settings": {"index.number_of_shards": 1}}
|
|
|
|
|
|
|
|
# create the repository
|
|
|
|
PUT /_snapshot/my_repository
|
|
|
|
{"type": "fs","settings": {"location": "recovery_asciidoc" }}
|
|
|
|
|
|
|
|
# snapshot the index
|
|
|
|
PUT /_snapshot/my_repository/snap_1?wait_for_completion=true
|
2019-09-19 10:05:51 -04:00
|
|
|
{"indices": "index1"}
|
2017-08-28 05:23:09 -04:00
|
|
|
|
|
|
|
# delete the index
|
|
|
|
DELETE index1
|
|
|
|
|
|
|
|
# and restore the snapshot
|
|
|
|
POST /_snapshot/my_repository/snap_1/_restore?wait_for_completion=true
|
|
|
|
|
|
|
|
--------------------------------------------------
|
|
|
|
|
2019-09-06 09:22:08 -04:00
|
|
|
[source,console-result]
|
2017-08-28 05:23:09 -04:00
|
|
|
--------------------------------------------------
|
|
|
|
{
|
|
|
|
"snapshot": {
|
|
|
|
"snapshot": "snap_1",
|
|
|
|
"indices": [
|
|
|
|
"index1"
|
|
|
|
],
|
|
|
|
"shards": {
|
|
|
|
"total": 1,
|
|
|
|
"failed": 0,
|
|
|
|
"successful": 1
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
//////////////////////////
|
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
[source,console]
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
2016-09-01 13:05:22 -04:00
|
|
|
GET /_recovery?human
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
2017-08-28 05:23:09 -04:00
|
|
|
// TEST[continued]
|
2014-02-18 17:28:49 -05:00
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
The API returns the following response:
|
2019-09-06 16:09:09 -04:00
|
|
|
|
|
|
|
[source,console-result]
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
|
|
|
{
|
|
|
|
"index1" : {
|
|
|
|
"shards" : [ {
|
|
|
|
"id" : 0,
|
2015-02-10 02:54:03 -05:00
|
|
|
"type" : "SNAPSHOT",
|
|
|
|
"stage" : "INDEX",
|
2014-02-18 17:28:49 -05:00
|
|
|
"primary" : true,
|
|
|
|
"start_time" : "2014-02-24T12:15:59.716",
|
2015-02-10 02:54:03 -05:00
|
|
|
"start_time_in_millis": 1393244159716,
|
2017-08-28 05:23:09 -04:00
|
|
|
"stop_time" : "0s",
|
|
|
|
"stop_time_in_millis" : 0,
|
2016-09-01 13:05:22 -04:00
|
|
|
"total_time" : "2.9m",
|
2014-02-18 17:28:49 -05:00
|
|
|
"total_time_in_millis" : 175576,
|
|
|
|
"source" : {
|
|
|
|
"repository" : "my_repository",
|
|
|
|
"snapshot" : "my_snapshot",
|
2017-08-28 05:23:09 -04:00
|
|
|
"index" : "index1",
|
2018-12-14 05:39:23 -05:00
|
|
|
"version" : "{version}",
|
|
|
|
"restoreUUID": "PDh1ZAOaRbiGIVtCvZOMww"
|
2014-02-18 17:28:49 -05:00
|
|
|
},
|
|
|
|
"target" : {
|
|
|
|
"id" : "ryqJ5lO5S4-lSFbGntkEkg",
|
2017-08-28 05:23:09 -04:00
|
|
|
"host" : "my.fqdn",
|
|
|
|
"transport_address" : "my.fqdn",
|
2014-02-18 17:28:49 -05:00
|
|
|
"ip" : "10.0.1.7",
|
|
|
|
"name" : "my_es_node"
|
|
|
|
},
|
|
|
|
"index" : {
|
2015-02-10 02:54:03 -05:00
|
|
|
"size" : {
|
2016-09-01 13:05:22 -04:00
|
|
|
"total" : "75.4mb",
|
2015-02-10 02:54:03 -05:00
|
|
|
"total_in_bytes" : 79063092,
|
|
|
|
"reused" : "0b",
|
|
|
|
"reused_in_bytes" : 0,
|
|
|
|
"recovered" : "65.7mb",
|
|
|
|
"recovered_in_bytes" : 68891939,
|
|
|
|
"percent" : "87.1%"
|
|
|
|
},
|
2014-02-18 17:28:49 -05:00
|
|
|
"files" : {
|
|
|
|
"total" : 73,
|
|
|
|
"reused" : 0,
|
|
|
|
"recovered" : 69,
|
|
|
|
"percent" : "94.5%"
|
|
|
|
},
|
2015-02-10 02:54:03 -05:00
|
|
|
"total_time" : "0s",
|
2017-08-28 05:23:09 -04:00
|
|
|
"total_time_in_millis" : 0,
|
|
|
|
"source_throttle_time" : "0s",
|
|
|
|
"source_throttle_time_in_millis" : 0,
|
|
|
|
"target_throttle_time" : "0s",
|
|
|
|
"target_throttle_time_in_millis" : 0
|
2014-02-18 17:28:49 -05:00
|
|
|
},
|
|
|
|
"translog" : {
|
|
|
|
"recovered" : 0,
|
2015-03-07 10:50:32 -05:00
|
|
|
"total" : 0,
|
|
|
|
"percent" : "100.0%",
|
|
|
|
"total_on_start" : 0,
|
2015-02-10 02:54:03 -05:00
|
|
|
"total_time" : "0s",
|
2016-09-01 13:05:22 -04:00
|
|
|
"total_time_in_millis" : 0,
|
2014-02-18 17:28:49 -05:00
|
|
|
},
|
2017-08-28 05:23:09 -04:00
|
|
|
"verify_index" : {
|
2015-02-10 02:54:03 -05:00
|
|
|
"check_index_time" : "0s",
|
|
|
|
"check_index_time_in_millis" : 0,
|
|
|
|
"total_time" : "0s",
|
2014-02-18 17:28:49 -05:00
|
|
|
"total_time_in_millis" : 0
|
|
|
|
}
|
|
|
|
} ]
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
2017-08-28 05:23:09 -04:00
|
|
|
// TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]
|
|
|
|
// TESTRESPONSE[s/: "[^"]*"/: $body.$_path/]
|
|
|
|
////
|
|
|
|
The TESTRESPONSE above replace all the fields values by the expected ones in the test,
|
|
|
|
because we don't really care about the field values but we want to check the fields names.
|
|
|
|
////
|
2014-02-18 17:28:49 -05:00
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
This response includes information
|
|
|
|
about a single index recovering a single shard.
|
|
|
|
The source of the recovery is a snapshot repository
|
|
|
|
and the target of the recovery is the `my_es_node` node.
|
|
|
|
|
|
|
|
The response also includes the number
|
|
|
|
and percentage of files and bytes recovered.
|
2014-02-18 17:28:49 -05:00
|
|
|
|
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
[[index-recovery-api-detailed-ex]]
|
|
|
|
===== Get detailed recovery information
|
|
|
|
|
|
|
|
To get a list of physical files in recovery,
|
|
|
|
set the `detailed` query parameter to `true`.
|
2014-02-18 17:28:49 -05:00
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
[source,console]
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
2016-09-01 13:05:22 -04:00
|
|
|
GET _recovery?human&detailed=true
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
2017-08-28 05:23:09 -04:00
|
|
|
// TEST[s/^/PUT index1\n{"settings": {"index.number_of_shards": 1}}\n/]
|
2014-02-18 17:28:49 -05:00
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
The API returns the following response:
|
2014-02-18 17:28:49 -05:00
|
|
|
|
2019-09-06 16:09:09 -04:00
|
|
|
[source,console-result]
|
2014-02-18 17:28:49 -05:00
|
|
|
--------------------------------------------------
|
|
|
|
{
|
|
|
|
"index1" : {
|
|
|
|
"shards" : [ {
|
|
|
|
"id" : 0,
|
2015-06-23 16:10:55 -04:00
|
|
|
"type" : "STORE",
|
2015-02-10 02:54:03 -05:00
|
|
|
"stage" : "DONE",
|
2014-02-18 17:28:49 -05:00
|
|
|
"primary" : true,
|
|
|
|
"start_time" : "2014-02-24T12:38:06.349",
|
2015-02-10 02:54:03 -05:00
|
|
|
"start_time_in_millis" : "1393245486349",
|
2014-02-18 17:28:49 -05:00
|
|
|
"stop_time" : "2014-02-24T12:38:08.464",
|
2015-02-10 02:54:03 -05:00
|
|
|
"stop_time_in_millis" : "1393245488464",
|
|
|
|
"total_time" : "2.1s",
|
2014-02-18 17:28:49 -05:00
|
|
|
"total_time_in_millis" : 2115,
|
|
|
|
"source" : {
|
|
|
|
"id" : "RGMdRc-yQWWKIBM4DGvwqQ",
|
2017-08-28 05:23:09 -04:00
|
|
|
"host" : "my.fqdn",
|
|
|
|
"transport_address" : "my.fqdn",
|
2014-02-18 17:28:49 -05:00
|
|
|
"ip" : "10.0.1.7",
|
|
|
|
"name" : "my_es_node"
|
|
|
|
},
|
|
|
|
"target" : {
|
|
|
|
"id" : "RGMdRc-yQWWKIBM4DGvwqQ",
|
2017-08-28 05:23:09 -04:00
|
|
|
"host" : "my.fqdn",
|
|
|
|
"transport_address" : "my.fqdn",
|
2014-02-18 17:28:49 -05:00
|
|
|
"ip" : "10.0.1.7",
|
|
|
|
"name" : "my_es_node"
|
|
|
|
},
|
|
|
|
"index" : {
|
2015-02-10 02:54:03 -05:00
|
|
|
"size" : {
|
|
|
|
"total" : "24.7mb",
|
|
|
|
"total_in_bytes" : 26001617,
|
|
|
|
"reused" : "24.7mb",
|
|
|
|
"reused_in_bytes" : 26001617,
|
|
|
|
"recovered" : "0b",
|
|
|
|
"recovered_in_bytes" : 0,
|
|
|
|
"percent" : "100.0%"
|
|
|
|
},
|
2014-02-18 17:28:49 -05:00
|
|
|
"files" : {
|
|
|
|
"total" : 26,
|
|
|
|
"reused" : 26,
|
2015-02-10 02:54:03 -05:00
|
|
|
"recovered" : 0,
|
2014-02-18 17:28:49 -05:00
|
|
|
"percent" : "100.0%",
|
|
|
|
"details" : [ {
|
|
|
|
"name" : "segments.gen",
|
|
|
|
"length" : 20,
|
|
|
|
"recovered" : 20
|
|
|
|
}, {
|
|
|
|
"name" : "_0.cfs",
|
|
|
|
"length" : 135306,
|
|
|
|
"recovered" : 135306
|
|
|
|
}, {
|
|
|
|
"name" : "segments_2",
|
|
|
|
"length" : 251,
|
|
|
|
"recovered" : 251
|
2017-08-28 05:23:09 -04:00
|
|
|
}
|
2014-02-18 17:28:49 -05:00
|
|
|
]
|
|
|
|
},
|
2015-02-10 02:54:03 -05:00
|
|
|
"total_time" : "2ms",
|
2017-08-28 05:23:09 -04:00
|
|
|
"total_time_in_millis" : 2,
|
|
|
|
"source_throttle_time" : "0s",
|
|
|
|
"source_throttle_time_in_millis" : 0,
|
|
|
|
"target_throttle_time" : "0s",
|
|
|
|
"target_throttle_time_in_millis" : 0
|
2014-02-18 17:28:49 -05:00
|
|
|
},
|
|
|
|
"translog" : {
|
|
|
|
"recovered" : 71,
|
2017-08-28 05:23:09 -04:00
|
|
|
"total" : 0,
|
|
|
|
"percent" : "100.0%",
|
|
|
|
"total_on_start" : 0,
|
2015-02-10 02:54:03 -05:00
|
|
|
"total_time" : "2.0s",
|
2014-02-18 17:28:49 -05:00
|
|
|
"total_time_in_millis" : 2025
|
|
|
|
},
|
2017-08-28 05:23:09 -04:00
|
|
|
"verify_index" : {
|
2014-02-18 17:28:49 -05:00
|
|
|
"check_index_time" : 0,
|
2017-08-28 05:23:09 -04:00
|
|
|
"check_index_time_in_millis" : 0,
|
2015-02-10 02:54:03 -05:00
|
|
|
"total_time" : "88ms",
|
2014-02-18 17:28:49 -05:00
|
|
|
"total_time_in_millis" : 88
|
|
|
|
}
|
|
|
|
} ]
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
2017-08-28 05:23:09 -04:00
|
|
|
// TESTRESPONSE[s/"source" : \{[^}]*\}/"source" : $body.$_path/]
|
2019-06-21 03:02:23 -04:00
|
|
|
// TESTRESPONSE[s/"details" : \[[^\]]*\]/"details" : $body.$_path/]
|
2017-08-28 05:23:09 -04:00
|
|
|
// TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]
|
|
|
|
// TESTRESPONSE[s/: "[^"]*"/: $body.$_path/]
|
|
|
|
////
|
|
|
|
The TESTRESPONSE above replace all the fields values by the expected ones in the test,
|
|
|
|
because we don't really care about the field values but we want to check the fields names.
|
|
|
|
It also removes the "details" part which is important in this doc but really hard to test.
|
|
|
|
////
|
2014-02-18 17:28:49 -05:00
|
|
|
|
2019-10-03 16:43:26 -04:00
|
|
|
The response includes a listing
|
|
|
|
of any physical files recovered
|
|
|
|
and their sizes.
|
|
|
|
|
|
|
|
The response also includes timings in milliseconds
|
|
|
|
of the various stages of recovery:
|
|
|
|
|
|
|
|
* Index retrieval
|
|
|
|
* Translog replay
|
|
|
|
* Index start time
|
|
|
|
|
|
|
|
This response indicates the recovery is `done`.
|
|
|
|
All recoveries,
|
|
|
|
whether ongoing or complete,
|
|
|
|
are kept in the cluster state
|
|
|
|
and may be reported on at any time.
|
|
|
|
|
|
|
|
To only return information about ongoing recoveries,
|
|
|
|
set the `active_only` query parameter to `true`.
|