2018-11-13 12:15:37 -05:00
|
|
|
[role="xpack"]
|
|
|
|
|
[testenv="platinum"]
|
|
|
|
|
[[ccr-requirements]]
|
|
|
|
|
=== Requirements for leader indices
|
|
|
|
|
|
|
|
|
|
beta[]
|
|
|
|
|
|
|
|
|
|
Cross-cluster replication works by replaying the history of individual write
|
|
|
|
|
operations that were performed on the shards of the leader index. This means that the
|
|
|
|
|
history of these operations needs to be retained on the leader shards so that
|
|
|
|
|
they can be pulled by the follower shard tasks. The underlying mechanism used to
|
|
|
|
|
retain these operations is _soft deletes_. A soft delete occurs whenever an
|
|
|
|
|
existing document is deleted or updated. By retaining these soft deletes up to
|
|
|
|
|
configurable limits, the history of operations can be retained on the leader
|
|
|
|
|
shards and made available to the follower shard tasks as it replays the history
|
|
|
|
|
of operations.
|
|
|
|
|
|
|
|
|
|
Soft deletes must be enabled for indices that you want to use as leader
|
|
|
|
|
indices. Enabling soft deletes requires the addition of some index settings at
|
|
|
|
|
index creation time. You must add these settings to your create index
|
|
|
|
|
requests or to the index templates that you use to manage the creation of new
|
|
|
|
|
indices.
|
|
|
|
|
|
|
|
|
|
IMPORTANT: This means that {ccr} can not be used on existing indices. If you have
|
|
|
|
|
existing data that you want to replicate from another cluster, you must
|
|
|
|
|
{ref}/docs-reindex.html[reindex] your data into a new index with soft deletes
|
|
|
|
|
enabled.
|
|
|
|
|
|
|
|
|
|
[float]
|
|
|
|
|
[[ccr-overview-soft-deletes]]
|
|
|
|
|
==== Soft delete settings
|
|
|
|
|
|
|
|
|
|
`index.soft_deletes.enabled`::
|
|
|
|
|
|
|
|
|
|
Whether or not soft deletes are enabled on the index. Soft deletes can only be
|
|
|
|
|
configured at index creation and only on indices created on or after 6.5.0. The
|
|
|
|
|
default value is `false`.
|
|
|
|
|
|
|
|
|
|
`index.soft_deletes.retention.operations`::
|
|
|
|
|
|
|
|
|
|
The number of soft deletes to retain. Soft deletes are collected during merges
|
|
|
|
|
on the underlying Lucene index yet retained up to the number of operations
|
|
|
|
|
configured by this setting. The default value is `0`.
|
|
|
|
|
|
2018-12-06 16:07:40 -05:00
|
|
|
For more information about index settings, see {ref}/index-modules.html[Index modules].
|
|
|
|
|
|
|
|
|
|
[float]
|
|
|
|
|
[[ccr-overview-beats]]
|
|
|
|
|
==== Setting soft deletes on indices created by APM Server or Beats
|
|
|
|
|
|
|
|
|
|
If you want to replicate indices created by APM Server or Beats, and are
|
|
|
|
|
allowing APM Server or Beats to manage index templates, you need to enable
|
|
|
|
|
soft deletes on the underlying index templates. To enable soft deletes on the
|
|
|
|
|
underlying index templates, add the following changes to the relevant APM Server
|
|
|
|
|
or Beats configuration file.
|
|
|
|
|
|
|
|
|
|
["source","yaml"]
|
|
|
|
|
----------------------------------------------------------------------
|
|
|
|
|
setup.template.overwrite: true
|
|
|
|
|
setup.template.settings:
|
|
|
|
|
index.soft_deletes.enabled: true
|
|
|
|
|
index.soft_deletes.retention.operations: 1024
|
|
|
|
|
----------------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
For additional information on controlling the index templates managed by APM
|
|
|
|
|
Server or Beats, see the relevant documentation on loading the Elasticsearch
|
|
|
|
|
index template.
|
