iSharkFly-Docs
/
opensearch-docs-cn
mirror of https://github.com/iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #370 from opensearch-project/cross-cluster-replication 

CCR updates from editing team
This commit is contained in:
Liz Snyder 2022-01-18 11:48:16 -08:00 committed by GitHub
parent 4ae53500c1 f1e3686e81
commit db625fc274
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 30 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
_replication-plugin/api.md
View File

@ -39,7 +39,7 @@ Options | Description | Type | Required
:--- | :--- |:--- |:--- |
`leader_alias` | The name of the cross-cluster connection. You define this alias when you [set up a cross-cluster connection]({{site.url}}{{site.baseurl}}/replication-plugin/get-started/#set-up-a-cross-cluster-connection). | `string` | Yes
`leader_index` | The index on the leader cluster that you want to replicate. | `string` | Yes
`use_roles` | The roles to use for all subsequent backend replication tasks between the indices. Specify a `leader_cluster_role` and `follower_cluster_role`. See [Map the leader and follower cluster roles]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/#map-the-leader-and-follower-cluster-roles). | `string` | If security plugin is enabled
`use_roles` | The roles to use for all subsequent backend replication tasks between the indexes. Specify a `leader_cluster_role` and `follower_cluster_role`. See [Map the leader and follower cluster roles]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/#map-the-leader-and-follower-cluster-roles). | `string` | If security plugin is enabled
#### Sample response
@ -144,13 +144,13 @@ GET /_plugins/_replication/<follower-index>/_status
```
To include shard replication details in the response, add the `&verbose=true` parameter.
The leader and follower checkpoint values begin as negative integers and reflect the shard count (-1 for one shard, -5 for five shards, and so on). The values increment toward positive integers with each change that you make. For example, when you make a change on the leader index, the `leader_checkpoint` becomes `0`. The `follower_checkpoint` is initially still `-1` until the follower index pulls the change from the leader, at which point it increments to `0`. If the values are the same, it means the indices are fully synced.
The leader and follower checkpoint values begin as negative integers and reflect the shard count (-1 for one shard, -5 for five shards, and so on). The values increment toward positive integers with each change that you make. For example, when you make a change on the leader index, the `leader_checkpoint` becomes `0`. The `follower_checkpoint` is initially still `-1` until the follower index pulls the change from the leader, at which point it increments to `0`. If the values are the same, it means the indexes are fully synced.
## Get leader cluster stats
Introduced 1.1
{: .label .label-purple }
Gets information about replicated leader indices on a specified cluster.
Gets information about replicated leader indexes on a specified cluster.
#### Request
@ -197,7 +197,7 @@ GET /_plugins/_replication/leader_stats
Introduced 1.1
{: .label .label-purple }
Gets information about follower (syncing) indices on a specified cluster.
Gets information about follower (syncing) indexes on a specified cluster.
#### Request
@ -318,7 +318,7 @@ PUT /_plugins/_replication/<follower-index>/_update
Introduced 1.1
{: .label .label-purple }
Automatically starts replication on indices matching a specified pattern. If a new index on the leader cluster matches the pattern, OpenSearch automatically creates a follower index and begins replication. You can also use this API to update existing replication rules.
Automatically starts replication on indexes matching a specified pattern. If a new index on the leader cluster matches the pattern, OpenSearch automatically creates a follower index and begins replication. You can also use this API to update existing replication rules.
Send this request to the follower cluster.
@ -346,8 +346,8 @@ Options | Description | Type | Required
:--- | :--- |:--- |:--- |
`leader_alias` | The name of the cross-cluster connection. You define this alias when you [set up a cross-cluster connection]({{site.url}}{{site.baseurl}}/replication-plugin/get-started/#set-up-a-cross-cluster-connection). | `string` | Yes
`name` | A name for the auto-follow pattern. | `string` | Yes
`pattern` | An array of index patterns to match against indices in the specified leader cluster. Supports wildcard characters. For example, `leader-*`. | `string` | Yes
`use_roles` | The roles to use for all subsequent backend replication tasks between the indices. Specify a `leader_cluster_role` and `follower_cluster_role`. See [Map the leader and follower cluster roles]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/#map-the-leader-and-follower-cluster-roles). | `string` | If security plugin is enabled
`pattern` | An array of index patterns to match against indexes in the specified leader cluster. Supports wildcard characters. For example, `leader-*`. | `string` | Yes
`use_roles` | The roles to use for all subsequent backend replication tasks between the indexes. Specify a `leader_cluster_role` and `follower_cluster_role`. See [Map the leader and follower cluster roles]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/#map-the-leader-and-follower-cluster-roles). | `string` | If security plugin is enabled
#### Sample response
@ -361,7 +361,7 @@ Options | Description | Type | Required
Introduced 1.1
{: .label .label-purple }
Deletes the specified replication rule. This operation prevents any new indices from being replicated but does not stop existing replication that the rule has already initiated. Replicated indices remain read-only until you stop replication.
Deletes the specified replication rule. This operation prevents any new indexes from being replicated but does not stop existing replication that the rule has already initiated. Replicated indexes remain read-only until you stop replication.
Send this request to the follower cluster.

10
_replication-plugin/auto-follow.md
View File

@ -8,7 +8,7 @@ has_children: false
# Auto-follow for cross-cluster replication
Auto-follow lets you automatically replicate indices created on the leader cluster based on matching patterns. When you create an index on the leader cluster with a name that matches a specified pattern (for example, `index-01*`), a corresponding follower index is automatically created on the follower cluster.
Auto-follow lets you automatically replicate indexes created on the leader cluster based on matching patterns. When you create an index on the leader cluster with a name that matches a specified pattern (for example, `index-01*`), a corresponding follower index is automatically created on the follower cluster.
You can configure multiple replication rules for a single cluster. The patterns currently only support wildcard matching.
@ -18,11 +18,11 @@ You need to [set up a cross-cluster connection]({{site.url}}{{site.baseurl}}/rep
## Permissions
If the security plugin is enabled, non-admin users need to be mapped to the appropriate permissions in order to perform replication actions. For index and cluster-level permissions requirements, see [Cross-cluster replication permissions]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/).
If the security plugin is enabled, make sure that non-admin users are mapped to the appropriate permissions so they can perform replication actions. For index and cluster-level permissions requirements, see [Cross-cluster replication permissions]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/).
## Get started with auto-follow
Replication rules are a collection of patterns that you create against a single remote cluster. When you create a replication rule, it automatically starts replicating any *new* indices that match the pattern, but does not replicate matching indices that were previously created.
Replication rules are a collection of patterns that you create against a single remote cluster. When you create a replication rule, it automatically starts replicating any *new* indexes that match the pattern, but does not replicate matching indexes that were previously created.
Create a replication rule on the follower cluster:
@ -63,7 +63,7 @@ yellow open movies-0001 kHOxYYHxRMeszLjTD9rvSQ 1 1 0
## Retrieve replication rules
To retrieve a list of existing replication rules configured on a cluster, send the following request:
To retrieve a list of existing replication rules that are configured on a cluster, send the following request:
```bash
curl -XGET -u 'admin:admin' -k 'https://localhost:9200/_plugins/_replication/autofollow_stats'
@ -102,5 +102,5 @@ curl -XDELETE -k -H 'Content-Type: application/json' -u 'admin:admin' 'https://l
}'
```
OpenSearch stops replicating *new* indices that match the pattern, but existing indices that the rule previously created remain read-only and continue to replicate. If you need to stop existing replication activity and open the indices up for writes, use the [stop replication API operation]({{site.url}}{{site.baseurl}}/replication-plugin/api/#stop-replication).
OpenSearch stops replicating *new* indexes that match the pattern, but existing indexes that the rule previously created remain read-only and continue to replicate. If you need to stop existing replication activity and open the indexes up for writes, use the [stop replication API operation]({{site.url}}{{site.baseurl}}/replication-plugin/api/#stop-replication).

20
_replication-plugin/get-started.md
View File

@ -6,7 +6,7 @@ nav_order: 10
# Get started with cross-cluster replication
With cross-cluster replication, you index data to a leader index, and OpenSearch replicates that data to one or more read-only follower indices. All subsequent operations on the leader are replicated on the follower, such as creating, updating, or deleting documents.
With cross-cluster replication, you index data to a leader index, and OpenSearch replicates that data to one or more read-only follower indexes. All subsequent operations on the leader are replicated on the follower, such as creating, updating, or deleting documents.
## Prerequisites
@ -22,9 +22,9 @@ Cross-cluster replication has the following prerequisites:
Make sure the security plugin is either enabled on both clusters or disabled on both clusters. If you disabled the security plugin, you can skip this section. However, we strongly recommend enabling the security plugin in production scenarios.
If the security plugin is enabled, non-admin users need to be mapped to the appropriate permissions in order to perform replication actions. For index and cluster-level permissions requirements, see [Cross-cluster replication permissions]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/).
If the security plugin is enabled, make sure that non-admin users are mapped to the appropriate permissions so they can perform replication actions. For index and cluster-level permissions requirements, see [Cross-cluster replication permissions]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/).
In addition, verify and add the Distinguished names (DNs) of each follower cluster node on the leader cluster to allow connections from the followers to the leader.
In addition, verify and add the distinguished names (DNs) of each follower cluster node on the leader cluster to allow connections from the followers to the leader.
First, get the node's DN from each follower cluster:
@ -43,7 +43,7 @@ curl -XGET -k -u 'admin:admin' 'https://localhost:9200/_plugins/_security/api/ss
}
```
Then verify that it's part of the leader cluster configuration in `opensearch.yml`, otherwise add it under the following setting:
Then verify that it's part of the leader cluster configuration in `opensearch.yml`. Otherwise, add it under the following setting:
```yaml
plugins.security.nodes_dn:
@ -52,7 +52,7 @@ plugins.security.nodes_dn:
```
## Example setup
Save this sample file as `docker-compose.yml` and run `docker-compose up` to start two single-node clusters on the same network:
To start two single-node clusters on the same network, save this sample file as `docker-compose.yml` and run `docker-compose up`:
```yml
version: '3'
@ -184,7 +184,7 @@ curl -XPUT -k -H 'Content-Type: application/json' -u 'admin:admin' 'https://loca
If the security plugin is disabled, omit the `use_roles` parameter. If it's enabled, however, you must specify the leader and follower cluster roles that OpenSearch will use to authenticate the request. This example uses `all_access` for simplicity, but we recommend creating a replication user on each cluster and [mapping it accordingly]({{site.url}}{{site.baseurl}}/replication-plugin/permissions/#map-the-leader-and-follower-cluster-roles).
{: .tip }
This command creates an identical read-only index named `follower-01` on the follower cluster that continuously stays updated with changes to the `leader-01` index on the leader cluster. Starting replication creates a follower index from scratch; you can't convert an existing index to a follower index.
This command creates an identical read-only index named `follower-01` on the follower cluster that continuously stays updated with changes to the `leader-01` index on the leader cluster. Starting replication creates a follower index from scratch -- you can't convert an existing index to a follower index.
## Confirm replication
@ -207,9 +207,9 @@ curl -XGET -k -u 'admin:admin' 'https://localhost:9200/_plugins/_replication/fol
}
```
Possible statuses are `SYNCING`, `BOOTSTRAPING`, `PAUSED`, and `REPLICATION NOT IN PROGRESS`.
Possible statuses are `SYNCING`, `BOOTSTRAPPING`, `PAUSED`, and `REPLICATION NOT IN PROGRESS`.
The leader and follower checkpoint values begin as negative numbers and reflect the shard count (-1 for one shard, -5 for five shards, and so on). The values increment with each change and illustrate how many updates the follower is behind the leader. If the indices are fully synced, the values are the same.
The leader and follower checkpoint values begin as negative numbers and reflect the shard count (-1 for one shard, -5 for five shards, and so on). The values increment with each change and illustrate how many updates the follower is behind the leader. If the indexes are fully synced, the values are the same.
To confirm that replication is actually happening, add a document to the leader index:
@ -244,7 +244,7 @@ You can temporarily pause replication of an index if you need to remediate issue
curl -XPOST -k -H 'Content-Type: application/json' -u 'admin:admin' 'https://localhost:9200/_plugins/_replication/follower-01/_pause?pretty' -d '{}'
```
To confirm replication is paused, get the status:
To confirm that replication is paused, get the status:
```bash
curl -XGET -k -u 'admin:admin' 'https://localhost:9200/_plugins/_replication/follower-01/_status?pretty'
@ -270,7 +270,7 @@ Note that you can't resume replication after it's been paused for more than 12 h
## Stop replication
Terminate replication of a specified index from the follower cluster:
When you no longer need to replicate an index, terminate replication from the follower cluster:
```bash
curl -XPOST -k -H 'Content-Type: application/json' -u 'admin:admin' 'https://localhost:9200/_plugins/_replication/follower-01/_stop?pretty' -d '{}'

8
_replication-plugin/index.md
View File

@ -9,14 +9,14 @@ redirect_from:
# Cross-cluster replication
The cross-cluster replication plugin lets you replicate indices, mappings, and metadata from one OpenSearch cluster to another. Cross-cluster replication has the following benefits:
- By replicating your indices, you ensure that you can continue to handle search requests in the event of an outage.
- Replicating data across geographically distant data centers minimizes the distance between the data and the application server, reducing expensive latencies.
The cross-cluster replication plugin lets you replicate indexes, mappings, and metadata from one OpenSearch cluster to another. Cross-cluster replication has the following benefits:
- By replicating your indexes, you ensure that you can continue to handle search requests if there's an outage.
- Replicating data across geographically distant data centers minimizes the distance between the data and the application server. This reduces expensive latencies.
- You can replicate data from multiple smaller clusters to a centralized reporting cluster, which is useful when it's inefficient to query across a large network.
Replication follows an active-passive model where the follower index (where the data is replicated) pulls data from the leader (remote) index.
The replication plugin supports replication of indices using wildcard pattern matching and provides commands to pause, resume, and stop replication. Once replication starts on an index, it initiates persistent background tasks on all primary shards on the follower cluster, which continuously poll corresponding shards from the leader cluster for updates.
The replication plugin supports replication of indexes using wildcard pattern matching and provides commands to pause, resume, and stop replication. Once replication starts on an index, it initiates persistent background tasks on all primary shards on the follower cluster, which continuously poll corresponding shards from the leader cluster for updates.
You can use the replication plugin with the security plugin to encrypt cross-cluster traffic with node-to-node encryption and control access to replication activities.

6
_replication-plugin/settings.md
View File

@ -25,12 +25,12 @@ These settings manage the resources consumed by remote recoveries. We don’t re
Setting | Default | Description
:--- | :--- | :---
`plugins.replication.follower.index.recovery.chunk_size` | 10MB | The chunk size requested by the follower cluster during file transfer. Specify the chunk size as a value and unit, for example, 10MB, 5KB. See [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).
`plugins.replication.follower.index.recovery.chunk_size` | 10 MB | The chunk size requested by the follower cluster during file transfer. Specify the chunk size as a value and unit, for example, 10 MB, 5 KB. See [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).
`plugins.replication.follower.index.recovery.max_concurrent_file_chunks` | 4 | The number of file chunk requests that can be sent in parallel for each recovery.
`plugins.replication.follower.index.ops_batch_size` | 5000 | The number of operations that can be fetched at a time during the syncing phase of replication.
`plugins.replication.follower.concurrent_readers_per_shard` | 2 | The number of concurrent requests from the follower cluster per shard during the syncing phase of replication.
`plugins.replication.autofollow.fetch_poll_interval` | 30s | How often auto-follow tasks poll the leader cluster for new matching indices.
`plugins.replication.autofollow.fetch_poll_interval` | 30s | How often auto-follow tasks poll the leader cluster for new matching indexes.
`plugins.replication.follower.metadata_sync_interval` | 60s | How often the follower cluster polls the leader cluster for updated index metadata.
`plugins.replication.translog.retention_lease.pruning.enabled` | true | If enabled, prunes the translog based on retention leases on the leader index.
`plugins.replication.translog.retention_size` | 512MB | Controls the size of the translog on the leader index.
`plugins.replication.translog.retention_size` | 512 MB | Controls the size of the translog on the leader index.