282 lines
8.2 KiB
Plaintext
282 lines
8.2 KiB
Plaintext
[[indices-synced-flush-api]]
|
|
=== Synced flush API
|
|
++++
|
|
<titleabbrev>Synced flush</titleabbrev>
|
|
++++
|
|
|
|
deprecated::[7.6, synced-flush is deprecated and will be removed in 8.0.
|
|
Use <<indices-flush,flush>> instead. A <<indices-flush,flush>> has the
|
|
same effect as a synced flush on Elasticsearch 7.6 or later]
|
|
|
|
Performs a synced flush on one or more indices.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
POST /twitter/_flush/synced
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
|
|
|
|
[[synced-flush-api-request]]
|
|
==== {api-request-title}
|
|
|
|
`POST /<index>/flush/synced`
|
|
|
|
`GET /<index>/flush/synced`
|
|
|
|
`POST /flush/synced`
|
|
|
|
`GET /flush/synced`
|
|
|
|
|
|
[[synced-flush-api-desc]]
|
|
==== {api-description-title}
|
|
|
|
[[synced-flush-using-api]]
|
|
===== Use the synced flush API
|
|
|
|
Use the synced flush API to manually initiate a synced flush.
|
|
This can be useful for a planned cluster restart where
|
|
you can stop indexing but don't want to wait for 5 minutes until all indices
|
|
are marked as inactive and automatically sync-flushed.
|
|
|
|
You can request a synced flush even if there is ongoing indexing activity, and
|
|
{es} will perform the synced flush on a "best-effort" basis: shards that do not
|
|
have any ongoing indexing activity will be successfully sync-flushed, and other
|
|
shards will fail to sync-flush. The successfully sync-flushed shards will have
|
|
faster recovery times as long as the `sync_id` marker is not removed by a
|
|
subsequent flush.
|
|
|
|
|
|
[[synced-flush-overview]]
|
|
===== Synced flush overview
|
|
|
|
{es} keeps track of which shards have received indexing activity recently, and
|
|
considers shards that have not received any indexing operations for 5 minutes to
|
|
be inactive.
|
|
|
|
When a shard becomes inactive {es} performs a special kind of flush
|
|
known as a *synced flush*. A synced flush performs a normal
|
|
<<indices-flush,flush>> on each replica of the shard, and then adds a marker known
|
|
as the `sync_id` to each replica to indicate that these copies have identical
|
|
Lucene indices. Comparing the `sync_id` markers of the two copies is a very
|
|
efficient way to check whether they have identical contents.
|
|
|
|
When allocating shard replicas, {es} must ensure that each replica contains the
|
|
same data as the primary. If the shard copies have been synced-flushed and the
|
|
replica shares a `sync_id` with the primary then {es} knows that the two copies
|
|
have identical contents. This means there is no need to copy any segment files
|
|
from the primary to the replica, which saves a good deal of time during
|
|
recoveries and restarts.
|
|
|
|
This is particularly useful for clusters having lots of indices which are very
|
|
rarely updated, such as with time-based indices. Without the synced flush
|
|
marker, recovery of this kind of cluster would be much slower.
|
|
|
|
|
|
[[synced-flush-sync-id-markers]]
|
|
===== Check for `sync_id` markers
|
|
|
|
To check whether a shard has a `sync_id` marker or not, look for the `commit`
|
|
section of the shard stats returned by the <<indices-stats,indices stats>> API:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
GET /twitter/_stats?filter_path=**.commit&level=shards <1>
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
|
|
<1> `filter_path` is used to reduce the verbosity of the response, but is entirely optional
|
|
|
|
The API returns the following response:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"indices": {
|
|
"twitter": {
|
|
"shards": {
|
|
"0": [
|
|
{
|
|
"commit" : {
|
|
"id" : "3M3zkw2GHMo2Y4h4/KFKCg==",
|
|
"generation" : 3,
|
|
"user_data" : {
|
|
"translog_uuid" : "hnOG3xFcTDeoI_kvvvOdNA",
|
|
"history_uuid" : "XP7KDJGiS1a2fHYiFL5TXQ",
|
|
"local_checkpoint" : "-1",
|
|
"translog_generation" : "2",
|
|
"max_seq_no" : "-1",
|
|
"sync_id" : "AVvFY-071siAOuFGEO9P", <1>
|
|
"max_unsafe_auto_id_timestamp" : "-1",
|
|
"min_retained_seq_no" : "0"
|
|
},
|
|
"num_docs" : 0
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
<1> the `sync id` marker
|
|
|
|
NOTE: The `sync_id` marker is removed as soon as the shard is flushed again, and
|
|
{es} may trigger an automatic flush of a shard at any time if there are
|
|
unflushed operations in the shard's translog. In practice this means that one
|
|
should consider any indexing operation on an index as having removed its
|
|
`sync_id` markers.
|
|
|
|
|
|
[[synced-flush-api-path-params]]
|
|
==== {api-path-parms-title}
|
|
|
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=index]
|
|
+
|
|
To sync-flush all indices,
|
|
omit this parameter
|
|
or use a value of `_all` or `*`.
|
|
|
|
|
|
[[synced-flush-api-query-params]]
|
|
==== {api-query-parms-title}
|
|
|
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
|
|
|
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
|
|
+
|
|
Defaults to `open`.
|
|
|
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
|
|
|
|
|
|
[[synced-flush-api-response-codes]]
|
|
==== {api-response-codes-title}
|
|
|
|
`200`::
|
|
All shards successfully sync-flushed.
|
|
|
|
`409`::
|
|
A replica shard failed to sync-flush.
|
|
|
|
|
|
[[synced-flush-api-example]]
|
|
==== {api-examples-title}
|
|
|
|
|
|
[[synced-flush-api-specific-ex]]
|
|
===== Sync-flush a specific index
|
|
|
|
[source,console]
|
|
----
|
|
POST /kimchy/_flush/synced
|
|
----
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
|
|
|
|
[[synced-flush-api-multi-ex]]
|
|
===== Synch-flush several indices
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
POST /kimchy,elasticsearch/_flush/synced
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
|
|
|
|
[[synced-flush-api-all-ex]]
|
|
===== Sync-flush all indices
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
POST /_flush/synced
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
|
|
The response contains details about how many shards were successfully
|
|
sync-flushed and information about any failure.
|
|
|
|
The following response indicates two shards
|
|
and one replica shard
|
|
successfully sync-flushed:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"_shards": {
|
|
"total": 2,
|
|
"successful": 2,
|
|
"failed": 0
|
|
},
|
|
"twitter": {
|
|
"total": 2,
|
|
"successful": 2,
|
|
"failed": 0
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
|
|
The following response indicates one shard group failed
|
|
due to pending operations:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"_shards": {
|
|
"total": 4,
|
|
"successful": 2,
|
|
"failed": 2
|
|
},
|
|
"twitter": {
|
|
"total": 4,
|
|
"successful": 2,
|
|
"failed": 2,
|
|
"failures": [
|
|
{
|
|
"shard": 1,
|
|
"reason": "[2] ongoing operations on primary"
|
|
}
|
|
]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|
|
|
|
Sometimes the failures are specific to a shard replica. The copies that failed
|
|
will not be eligible for fast recovery but those that succeeded still will be.
|
|
This case is reported as follows:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"_shards": {
|
|
"total": 4,
|
|
"successful": 1,
|
|
"failed": 1
|
|
},
|
|
"twitter": {
|
|
"total": 4,
|
|
"successful": 3,
|
|
"failed": 1,
|
|
"failures": [
|
|
{
|
|
"shard": 1,
|
|
"reason": "unexpected error",
|
|
"routing": {
|
|
"state": "STARTED",
|
|
"primary": false,
|
|
"node": "SZNr2J_ORxKTLUCydGX4zA",
|
|
"relocating_node": null,
|
|
"shard": 1,
|
|
"index": "twitter"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[skip: Synced flush can conflict with scheduled flushes in doc tests]
|