[DOCS] Updating CCR setup to be more tutorial focused (#62256) (#62499)

* Applying some initial changes.

* Updating intro and screenshots.

* Removing unnecessary links, streamlining content, and adding GIF.

* Adding what's next section.

* Removing what's next.

* Minor edits.

* Apply suggestions from code review

Co-authored-by: debadair <debadair@elastic.co>

* Incorporating review feedback.

* Moving CCR user privileges to another page, plus more edits.

* Apply suggestions from code review

Co-authored-by: debadair <debadair@elastic.co>

* Incorporating more review feedback.

* Adding TESTSETUP to fix build errors.

* Update docs/reference/ccr/getting-started.asciidoc

Co-authored-by: debadair <debadair@elastic.co>

* Swapping GIF for mp4 hosted on web team CMS.

* Removing GIF in favor of mp4.

Co-authored-by: debadair <debadair@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: debadair <debadair@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This commit is contained in:
Adam Locke 2020-09-17 09:50:21 -04:00 committed by GitHub
parent 3cf559bf9c
commit db9dd9f7e1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 174 additions and 175 deletions

View File

@ -1,90 +1,97 @@
[role="xpack"] [role="xpack"]
[testenv="platinum"] [testenv="platinum"]
[[ccr-getting-started]] [[ccr-getting-started]]
=== Set up {ccr} === Tutorial: Set up {ccr}
++++
<titleabbrev>Set up {ccr}</titleabbrev>
++++
////
[source,console]
----
PUT /server-metrics
{
"settings" : {
"index" : {
"number_of_shards" : 1,
"number_of_replicas" : 0
}
},
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"accept" : {
"type" : "long"
},
"deny" : {
"type" : "long"
},
"host" : {
"type" : "keyword"
},
"response" : {
"type" : "float"
},
"service" : {
"type" : "keyword"
},
"total" : {
"type" : "long"
}
}
}
}
----
// TESTSETUP
////
Use this guide to set up {ccr} (CCR) between clusters in two
datacenters. Replicating your data across datacenters provides several benefits:
* Brings data closer to your users or application server to reduce latency and
response time
* Provides your mission-critical applications with the tolerance to withstand datacenter or region outages
In this guide, you'll learn how to:
* Configure a <<modules-remote-clusters,remote cluster>> with a leader index
* Create a follower index on a local cluster
* Create an auto-follow pattern to automatically follow time series indices
that are periodically created in a remote cluster
You can manually create follower indices to replicate specific indices on a You can manually create follower indices to replicate specific indices on a
remote cluster, or configure auto-follow patterns to automatically create remote cluster, or configure auto-follow patterns to replicate rolling time series indices.
follower indices for new time series.
After the follower index is created, the video::https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt994089f5e841ad69/5f6265de6f40ab4648b5cf9b/ccr-setup-video-edited.mp4[width=700, height=500, options="autoplay,loop"]
<<ccr-remote-recovery, remote recovery>> process copies all of the Lucene
segment files from the remote cluster to the local cluster.
To set up {ccr}:
. <<ccr-getting-started-remote-cluster,Connect a local cluster to a remote cluster>>
. <<ccr-getting-started-leader-index,Identify the index (or time series indices) you want to replicate on the remote cluster>>
. <<ccr-enable-soft-deletes,Enable soft deletes on the leader index>>
. Manually create a follower index or create an auto-follow pattern:
* To replicate the leader index, <<ccr-getting-started-follower-index,manually create a follower index>>
* To automatically follow time series indices, <<ccr-getting-started-auto-follow,create an auto-follow pattern>>
[[ccr-getting-started-prerequisites]] [[ccr-getting-started-prerequisites]]
==== Prerequisites ==== Prerequisites
If the Elastic {security-features} are enabled in your local and remote To complete this tutorial, you need:
clusters, you need a user with appropriate authority to complete the steps
in this tutorial.
By default, you can complete the following steps as the built-in * A license on both clusters that includes {ccr}. {kibana-ref}/managing-licenses.html[Activate a free 30-day trial].
`elastic` user. However, you must <<get-started-built-in-users,set a password>> * The `read_ccr` cluster privilege and `monitor` and `read` privileges
for this user before proceeding. for the leader index on the remote cluster. <<stack-management-ccr-remote,Configure remote cluster privileges>>.
* The `manage_ccr` cluster privilege and `monitor`, `read`, `write` and
WARNING: If you are performing these steps in a production environment, do `manage_follow_index` privileges to configure remote clusters and follower
not use the `elastic` user. indices on the local cluster. <<stack-management-ccr-local,Configure local cluster privileges>>.
* An index on the remote cluster that contains the data you want to replicate.
Alternatively, you can assign the appropriate privileges to a user ID of your This tutorial uses the sample eCommerce orders data set.
choice. On the remote cluster that contains the leader index, a user must have {kibana-ref}/get-started.html#gs-get-data-into-kibana[Load sample data].
the `read_ccr` cluster privilege and `monitor` and `read` privileges on the
leader index.
[source,yml]
--------------------------------------------------
ccr_user:
cluster:
- read_ccr
indices:
- names: [ 'leader-index' ]
privileges:
- monitor
- read
--------------------------------------------------
On the local cluster that contains the follower index, the same user will need
the `manage_ccr` cluster privilege and `monitor`, `read`, `write` and
`manage_follow_index` privileges on the follower index.
[source,yml]
--------------------------------------------------
ccr_user:
cluster:
- manage_ccr
indices:
- names: [ 'follower-index' ]
privileges:
- monitor
- read
- write
- manage_follow_index
--------------------------------------------------
If you are managing
<<ccr-getting-started-remote-cluster,connecting to the remote cluster>> using
the cluster update settings API, you will also need a user with the `all`
cluster privilege.
[[ccr-getting-started-remote-cluster]] [[ccr-getting-started-remote-cluster]]
==== Connect to a remote cluster ==== Connect to a remote cluster
Connect your local cluster to a To replicate an index on a remote cluster (Cluster A) to a local cluster (Cluster B), you configure Cluster A as a remote on Cluster B.
<<modules-remote-clusters,remote cluster>> to begin using cross-cluster
replication.
To configure a {kibana-ref}/working-remote-clusters.html[remote cluster], image::images/ccr-tutorial-clusters.png[ClusterA contains the leader index and ClusterB contains the follower index]
access {kib} and go to
*Management > Stack Management*. In the side navigation, select
*Remote Clusters*.
Add a remote cluster by specifying the IP address or host name, followed by the To configure a remote cluster from Stack Management in {kib}:
transport port of the remote cluster.
. Select *Remote Clusters* from the side navigation.
. Specify the IP address or host name of the remote cluster (ClusterB),
followed by the transport port of the remote cluster (defaults to `9300`). For
example, `192.168.1.1:9300`.
[role="screenshot"] [role="screenshot"]
image::images/ccr-add-remote-cluster.png["The Add remote clusters page in {kib}"] image::images/ccr-add-remote-cluster.png["The Add remote clusters page in {kib}"]
@ -153,96 +160,39 @@ remote cluster.
connected to. connected to.
==== ====
[[ccr-getting-started-leader-index]]
==== Create a leader index
To create a leader index, access {kib} on your _remote_ cluster and go to
*Management > Dev Tools*.
Copy the following example into the Console to create a leader index named
`server-metrics` in your remote cluster:
[%collapsible]
.Leader index example
====
[source,console]
--------------------------------------------------
PUT /server-metrics
{
"settings" : {
"index" : {
"number_of_shards" : 1,
"number_of_replicas" : 0
}
},
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"accept" : {
"type" : "long"
},
"deny" : {
"type" : "long"
},
"host" : {
"type" : "keyword"
},
"response" : {
"type" : "float"
},
"service" : {
"type" : "keyword"
},
"total" : {
"type" : "long"
}
}
}
}
--------------------------------------------------
// TEST[continued]
====
[[ccr-enable-soft-deletes]] [[ccr-enable-soft-deletes]]
==== Enable soft deletes on leader indices ==== Enable soft deletes on leader indices
<<ccr-leader-requirements,Soft deletes>> must be enabled for indices that you want to To follow an index, it must have been created with
use as leader indices. Soft deletes are enabled by default on new indices <<ccr-leader-requirements,soft deletes>> enabled. If the index doesnt have
created on or after {es} 7.0.0, so soft deletes enabled, you must reindex it and use the new index as the leader
*no further action is required if your cluster is running {es} 7.0.0 or later*. index. Soft deletes are enabled by default on new indices
created with {es} 7.0.0 and later.
include::{es-ref-dir}/ccr/index.asciidoc[tag=ccr-existing-indices-tag]
To enable soft deletes on indices created on versions of
{es} between 6.5.0 and 7.0.0, set <<ccr-index-soft-deletes,`index.soft_deletes.enabled`>> to `true`.
[[ccr-getting-started-follower-index]] [[ccr-getting-started-follower-index]]
==== Create a follower index ==== Create a follower index to replicate a specific index
When you create a {kibana-ref}/managing-cross-cluster-replication.html#_create_specific_follower_indices[follower index], you When you create a follower index, you reference the remote cluster and the
must reference the leader index in your remote cluster.
<<ccr-getting-started-remote-cluster,remote cluster>> and the
<<ccr-getting-started-leader-index,leader index>> that you created in the remote
cluster.
To create a follower index, access {kib} and go to To create a follower index from Stack Management in {kib}:
*Management > Stack Management*. In the side navigation, select
*Cross-Cluster Replication* and choose the *Follower Indices* tab.
. Choose the remote cluster containing the index you want to replicate, which . Select *Cross-Cluster Replication* in the side navigation and choose the
is `leader` if you are following the tutorial. *Follower Indices* tab.
. Enter the name of the leader index, which is `server-metrics` if you are . Choose the cluster (ClusterA) containing the leader index you want to
following the tutorial. replicate.
. Enter the name of the leader index, which is
`kibana_sample_data_ecommerce` if you are following the tutorial.
. Enter a name for your follower index, such as `follower-kibana-sample-data`.
image::images/ccr-add-follower-index.png["Adding a follower index named server-metrics in {kib}"] image::images/ccr-add-follower-index.png["Adding a follower index named server-metrics in {kib}"]
The follower index is initialized using the {es} initializes the follower using the
<<ccr-remote-recovery, remote recovery>> <<ccr-remote-recovery, remote recovery>>
process, which transfers the existing Lucene segment files from the leader process, which transfers the existing Lucene segment files from the leader
index to the follower index. The index status changes to *Paused*. When the index to the follower index. The index status changes to *Paused*. When the
remote recovery process is complete, the index following begins and the status remote recovery process is complete, the index following begins and the status
changes to *Active*. changes to *Active*.
When you index documents into your leader index, the documents are replicated When you index documents into your leader index, {es} replicates the documents
in the follower index. in the follower index.
[role="screenshot"] [role="screenshot"]
@ -252,9 +202,8 @@ image::images/ccr-follower-index.png["The Cross-Cluster Replication page in {kib
.API example .API example
==== ====
Use the <<ccr-put-follow,create follower API>> to create follower indices. Use the <<ccr-put-follow,create follower API>> to create follower indices.
When you create a follower index, you must reference the When you create a follower index, you must reference the remote cluster and the
<<ccr-getting-started-remote-cluster,remote cluster>> and the leader index that you created in the
<<ccr-getting-started-leader-index,leader index>> that you created in the
remote cluster. remote cluster.
When initiating the follower request, the response returns before the When initiating the follower request, the response returns before the
@ -304,37 +253,35 @@ POST /server-metrics-follower/_ccr/unfollow
==== ====
[[ccr-getting-started-auto-follow]] [[ccr-getting-started-auto-follow]]
==== Automatically create follower indices ==== Create an auto-follow pattern to replicate time series indices
Create <<ccr-auto-follow,auto-follow patterns>> to automatically follow time You use <<ccr-auto-follow,auto-follow patterns>> to automatically create new
series indices that are periodically created in a remote cluster (such as daily followers for rolling time series indices. Whenever the name of a new index on
{beats} indices). the remote cluster matches the auto-follow pattern, a corresponding follower
index is added to the local cluster.
With an auto-follow pattern, you reference the An auto-follow pattern specifies the remote cluster you want to replicate from,
<<ccr-getting-started-remote-cluster,remote cluster>> connected to your and one or more index patterns that specify the rolling time series indices you
local cluster. You must also specify a collection of patterns that match the want to replicate.
indices you want to automatically follow.
// tag::ccr-create-auto-follow-pattern-tag[] // tag::ccr-create-auto-follow-pattern-tag[]
To create follower indices from an {kibana-ref}/managing-cross-cluster-replication.html#_create_follower_indices_from_an_auto_follow_pattern[auto-follow pattern], To create an auto-follow pattern from Stack Management in {kib}:
access {kib} on your remote cluster and go to
*Management > Stack Management*. In the side navigation, select
*Cross Cluster Replication* and choose the *Auto-follow patterns* tab.
[role="screenshot"] . Select *Cross Cluster Replication* in the side navigation and choose the
image::images/auto-follow-patterns.png["The Auto-follow patterns page in {kib}"] *Auto-follow patterns* tab.
. Enter a name for the auto-follow pattern, such as `beats`.
* Enter a name for the auto-follow pattern. For this tutorial, enter `beats` . Choose the remote cluster that contains the index you want to replicate,
as the name. which in the example scenario is Cluster A.
* Choose the remote cluster containing the index you want to replicate, which . Enter one or more index patterns that identify the indices you want to
is `leader` if you are following the tutorial. replicate from the remote cluster. For example, enter
* Enter one or more index patterns that identify the indices you want to `metricbeat-* packetbeat-*` to automatically create followers for {metricbeat} and {packetbeat} indices.
replicate from the remote cluster. For this tutorial, enter . Enter *follower-* as the prefix to apply to the names of the follower indices so
`metricbeat-*,packetbeat-*` as the index pattern.
* Enter *copy-* as the prefix to apply to the names of the follower indices so
you can more easily identify replicated indices. you can more easily identify replicated indices.
As new indices matching these patterns are As new indices matching these patterns are
created, they are replicated to the follower indices. created on the remote, {es} automatically replicates them to local follower indices.
[role="screenshot"]
image::images/auto-follow-patterns.png["The Auto-follow patterns page in {kib}"]
// end::ccr-create-auto-follow-pattern-tag[] // end::ccr-create-auto-follow-pattern-tag[]

Binary file not shown.

After

Width:  |  Height:  |  Size: 357 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 162 KiB

After

Width:  |  Height:  |  Size: 159 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 273 KiB

After

Width:  |  Height:  |  Size: 280 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 279 KiB

After

Width:  |  Height:  |  Size: 279 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 174 KiB

After

Width:  |  Height:  |  Size: 189 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

View File

@ -5,6 +5,8 @@ include::built-in-roles.asciidoc[]
include::managing-roles.asciidoc[] include::managing-roles.asciidoc[]
include::stack-management.asciidoc[]
include::privileges.asciidoc[] include::privileges.asciidoc[]
include::document-level-security.asciidoc[] include::document-level-security.asciidoc[]

View File

@ -0,0 +1,50 @@
[role="xpack"]
[[stack-management]]
=== Granting access to Stack Management features
You <<defining-roles,define roles>> and set user privileges at different levels
to grant access to each of the Elastic Stack features.
[[stack-management-ccr]]
==== {ccr-cap}
The {ccr} user requires different cluster and index privileges on the remote
cluster and local cluster.
[[stack-management-ccr-remote]]
On the remote cluster that contains the leader index, the {ccr} user requires
`read_ccr` cluster privilege and `monitor` and `read` privileges on the
leader index.
[source,yml]
--------------------------------------------------
ccr_user:
cluster:
- read_ccr
indices:
- names: [ 'leader-index' ]
privileges:
- monitor
- read
--------------------------------------------------
[[stack-management-ccr-local]]
On the local cluster that contains the follower index, the {ccr} user requires the `manage_ccr` cluster privilege and `monitor`, `read`, `write` and
`manage_follow_index` privileges on the follower index.
[source,yml]
--------------------------------------------------
ccr_user:
cluster:
- manage_ccr
indices:
- names: [ 'follower-index' ]
privileges:
- monitor
- read
- write
- manage_follow_index
--------------------------------------------------
If you are managing
<<ccr-getting-started-remote-cluster,connecting to the remote cluster>> using
the cluster update settings API, you will also need a user with the `all`
cluster privilege.