From 8b2f22ee075836280a81486ff31210f383e2c775 Mon Sep 17 00:00:00 2001 From: Jason Tedor Date: Tue, 13 Nov 2018 12:15:37 -0500 Subject: [PATCH] Introduce CCR overview (#35436) This commit introduces a basic overview for cross-cluster replication to the docs. Co-authored-by: "lcawl " --- docs/reference/ccr/auto-follow.asciidoc | 32 +++++++ docs/reference/ccr/index.asciidoc | 17 +++- docs/reference/ccr/overview.asciidoc | 107 ++++++++++++++++++++++- docs/reference/ccr/requirements.asciidoc | 45 ++++++++++ 4 files changed, 198 insertions(+), 3 deletions(-) create mode 100644 docs/reference/ccr/auto-follow.asciidoc create mode 100644 docs/reference/ccr/requirements.asciidoc diff --git a/docs/reference/ccr/auto-follow.asciidoc b/docs/reference/ccr/auto-follow.asciidoc new file mode 100644 index 00000000000..77fe26367c3 --- /dev/null +++ b/docs/reference/ccr/auto-follow.asciidoc @@ -0,0 +1,32 @@ +[role="xpack"] +[testenv="platinum"] +[[ccr-overview-auto-follow]] +=== Automatically following indices + +beta[] + +In time series use cases where you want to follow new indices that are +periodically created (such as daily Beats indices), manually configuring follower +indices for each new leader index can be an operational burden. The auto-follow +functionality in {ccr} is aimed at easing this burden. With the auto-follow +functionality, you can specify that new indices in a remote cluster that have a +name that matches a pattern are automatically followed. + +==== Managing auto-follow patterns + +You can add a new auto-follow pattern configuration with the +{ref}/ccr-put-auto-follow-pattern.html[create auto-follow pattern API]. When you create +a new auto-follow pattern configuration, you are configuring a collection of +patterns against a single remote cluster. Any time a new index with a name that +matches one of the patterns in the collection is created in the remote cluster, +a follower index is configured in the local cluster. The follower index uses the +new index as its leader index. + +You can inspect all configured auto-follow pattern collections with the +{ref}/ccr-get-auto-follow-pattern.html[get auto-follow pattern API]. To delete a +configured auto-follow pattern collection, use the +{ref}ccr-delete-auto-follow-pattern.html[delete auto-follow pattern API]. + +Since auto-follow functionality is handled automatically in the background on +your behalf, error reporting is done through logs on the elected master node +and through the {ref}/ccr-get-stats.html[{ccr} stats API]. diff --git a/docs/reference/ccr/index.asciidoc b/docs/reference/ccr/index.asciidoc index e802286af7c..aefa0ec7e8b 100644 --- a/docs/reference/ccr/index.asciidoc +++ b/docs/reference/ccr/index.asciidoc @@ -8,10 +8,23 @@ beta[] -* <> -* <> +The {ccr} (CCR) feature enables replication of indices in remote clusters to a +local cluster. This functionality can be used in some common production use cases: + +* Disaster recovery in case a primary cluster fails. A secondary cluster can + serve as a hot backup +* Geo-proximity so that reads can be served locally + +This guide provides an overview of {ccr}: + +* <> +* <> +* <> +* <> -- include::overview.asciidoc[] +include::requirements.asciidoc[] +include::auto-follow.asciidoc[] include::getting-started.asciidoc[] diff --git a/docs/reference/ccr/overview.asciidoc b/docs/reference/ccr/overview.asciidoc index c0ca8580f68..633da63288b 100644 --- a/docs/reference/ccr/overview.asciidoc +++ b/docs/reference/ccr/overview.asciidoc @@ -4,4 +4,109 @@ == Overview beta[] -This is the overview section of the {ccr} docs. + +Cross-cluster replication is done on an index-by-index basis. Replication is +configured at the index level. For each configured replication there is a +replication source index called the _leader index_ and a replication target +index called the _follower index_. + +Replication is active-passive. This means that while the leader index +can directly be written into, the follower index can not directly receive +writes. + +Replication is pull-based. This means that replication is driven by the +follower index. This simplifies state management on the leader index and means +that {ccr} does not interfere with indexing on the leader index. + +[float] +=== Configuring replication + +Replication can be configured in two ways: + +* Manually using the + {ref}/ccr-put-follow.html[create follower API] + +* Automatically using + <> + +NOTE: You must also <>. + +[float] +=== The mechanics of replication + +While replication is managed at the index level, replication is performed at the +shard level. When a follower index is created, it is automatically +configured to have an identical number of shards as the leader index. A follower +shard task in the follower index pulls from the corresponding leader shard in +the leader index by sending read requests for new operations. These read +requests can be served from any copy of the leader shard (primary or replicas). + +For each read request sent by the follower shard task, if there are new +operations available on the leader shard, the leader shard responds with +operations limited by the read parameters that you established when you +configured the follower index. If there are no new operations available on the +leader shard, the leader shard waits up to a configured timeout for new +operations. If new operations occur within that timeout, the leader shard +immediately responds with those new operations. Otherwise, if the timeout +elapses, the follower shard replies that there are no new operations. The +follower shard task updates some statistics and immediately sends another read +request to the leader shard. This ensures that the network connections between +the remote cluster and the local cluster are continually being used so as to +avoid forceful termination by an external source (such as a firewall). + +If a read request fails, the cause of the failure is inspected. If the +cause of the failure is deemed to be a failure that can be recovered from (for +example, a network failure), the follower shard task enters into a retry +loop. Otherwise, the follower shard task is paused and requires user +intervention before the it can be resumed with the +{ref}/ccr-post-resume-follow.html[resume follower API]. + +When operations are received by the follower shard task, they are placed in a +write buffer. The follower shard task manages this write buffer and submits +bulk write requests from this write buffer to the follower shard. The write +buffer and these write requests are managed by the write parameters that you +established when you configured the follower index. The write buffer serves as +back-pressure against read requests. If the write buffer exceeds its configured +limits, no additional read requests are sent by the follower shard task. The +follower shard task resumes sending read requests when the write buffer no +longer exceeds its configured limits. + +Mapping updates applied to the leader index are automatically retrieved +as-needed by the follower index. + +Settings updates applied to the leader index that are needed by the follower +index are automatically retried as-needed by the follower index. Not all +settings updates are needed by the follower index. For example, changing the +number of replicas on the leader index is not replicated by the follower index. + +NOTE: If you apply a non-dynamic settings change to the leader index that is +needed by the follower index, the follower index will go through a cycle of +closing itself, applying the settings update, and then re-opening itself. The +follower index will be unavailable for reads and not replicating writes +during this cycle. + +[float] +=== Inspecting the progress of replication + +You can inspect the progress of replication at the shard level with the +{ref}/ccr-get-follow-stats.html[get follower stats API]. This API gives you +insight into the read and writes managed by the follower shard task. It also +reports read exceptions that can be retried and fatal exceptions that require +user intervention. + +[float] +=== Pausing and resuming replication + +You can pause replication with the +{ref}/ccr-post-pause-follow.html[pause follower API] and then later resume +replication with the {ref}/ccr-post-resume-follow.html[resume follower API]. +Using these APIs in tandem enables you to adjust the read and write parameters +on the follower shard task if your initial configuration is not suitable for +your use case. + +[float] +=== Terminating replication + +You can terminate replication with the +{ref}/ccr-post-unfollow.html[unfollow API]. This API converts a follower index +to a regular (non-follower) index. diff --git a/docs/reference/ccr/requirements.asciidoc b/docs/reference/ccr/requirements.asciidoc new file mode 100644 index 00000000000..57f3f59eb35 --- /dev/null +++ b/docs/reference/ccr/requirements.asciidoc @@ -0,0 +1,45 @@ +[role="xpack"] +[testenv="platinum"] +[[ccr-requirements]] +=== Requirements for leader indices + +beta[] + +Cross-cluster replication works by replaying the history of individual write +operations that were performed on the shards of the leader index. This means that the +history of these operations needs to be retained on the leader shards so that +they can be pulled by the follower shard tasks. The underlying mechanism used to +retain these operations is _soft deletes_. A soft delete occurs whenever an +existing document is deleted or updated. By retaining these soft deletes up to +configurable limits, the history of operations can be retained on the leader +shards and made available to the follower shard tasks as it replays the history +of operations. + +Soft deletes must be enabled for indices that you want to use as leader +indices. Enabling soft deletes requires the addition of some index settings at +index creation time. You must add these settings to your create index +requests or to the index templates that you use to manage the creation of new +indices. + +IMPORTANT: This means that {ccr} can not be used on existing indices. If you have +existing data that you want to replicate from another cluster, you must +{ref}/docs-reindex.html[reindex] your data into a new index with soft deletes +enabled. + +[float] +[[ccr-overview-soft-deletes]] +==== Soft delete settings + +`index.soft_deletes.enabled`:: + +Whether or not soft deletes are enabled on the index. Soft deletes can only be +configured at index creation and only on indices created on or after 6.5.0. The +default value is `false`. + +`index.soft_deletes.retention.operations`:: + +The number of soft deletes to retain. Soft deletes are collected during merges +on the underlying Lucene index yet retained up to the number of operations +configured by this setting. The default value is `0`. + +For more information about index settings, see {ref}/index-modules.html[Index modules]. \ No newline at end of file