Add segement replication upgrade changes for 2.8 (#4187)
* Add segement replication upgrade changes for 2.8 Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add additional link Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Implemented tech review comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com>
This commit is contained in:
Naarcha-AWS
GitHub
parent
04929ec6df
commit
8c82ffe399
|
|
@ -93,6 +93,7 @@ A rolling upgrade is a great option if you want to keep your cluster operational
|
|||
|
||||
OpenSearch nodes cannot join a cluster if the cluster manager is running a newer version of OpenSearch than the node requesting membership. To avoid this issue, upgrade the cluster-manager-eligible nodes last.
|
||||
|
||||
|
||||
See [Rolling Upgrade]({{site.url}}{{site.baseurl}}/install-and-configure/upgrade-opensearch/rolling-upgrade/) for more information about the process.
|
||||
|
||||
### Cluster restart upgrade
|
||||
|
|
|
|||
|
|
@ -69,6 +69,7 @@ Review [Upgrading OpenSearch]({{site.url}}{{site.baseurl}}/upgrade-opensearch/in
|
|||
"transient" : { }
|
||||
}
|
||||
```
|
||||
|
||||
1. Perform a flush operation on the cluster to commit transaction log entries to the Lucene index:
|
||||
```json
|
||||
POST "/_flush?pretty"
|
||||
|
|
|
|||
|
|
@ -1,13 +1,13 @@
|
|||
---
|
||||
layout: default
|
||||
title: Segment replication back-pressure
|
||||
title: Segment replication backpressure
|
||||
nav_order: 75
|
||||
parent: Segment replication
|
||||
has_children: false
|
||||
grand_parent: Availability and Recovery
|
||||
---
|
||||
|
||||
## Segment replication backpressure
|
||||
# Segment replication backpressure
|
||||
|
||||
Segment replication backpressure is a shard-level rejection mechanism that dynamically rejects indexing requests as replica shards in your cluster fall behind primary shards. With segment replication backpressure, indexing requests are rejected when the percentage of stale shards in the replication group exceeds `MAX_ALLOWED_STALE_SHARDS` (50% by default). A replica is considered stale if it is behind the primary shard by the number of checkpoints that exceeds the `MAX_INDEXING_CHECKPOINTS` setting and its current replication lag is greater than the defined `MAX_REPLICATION_TIME_SETTING` field.
|
||||
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ Segment replication currently does not support the `wait_for` value in the `refr
|
|||
|
||||
For the best performance, we recommend enabling both of the following settings:
|
||||
|
||||
1. [Segment replication backpressure]({{site.url}}{{site.baseurl}}tuning-your-cluster/availability-and-recovery/segment-replication/backpressure/).
|
||||
1. [Segment replication backpressure]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/segment-replication/backpressure/).
|
||||
2. Balanced primary shard allocation:
|
||||
|
||||
```json
|
||||
|
|
@ -70,7 +70,6 @@ curl -X PUT "$host/_cluster/settings?pretty" -H 'Content-Type: application/json'
|
|||
When using segment replication, consider the following:
|
||||
|
||||
1. Enabling segment replication for an existing index requires [reindexing](https://github.com/opensearch-project/OpenSearch/issues/3685).
|
||||
1. Rolling upgrades are not currently supported. Full cluster restarts are required when upgrading indexes using segment replication. See [Issue 3881](https://github.com/opensearch-project/OpenSearch/issues/3881).
|
||||
1. [Cross-cluster replication](https://github.com/opensearch-project/OpenSearch/issues/4090) does not currently use segment replication to copy between clusters.
|
||||
1. Segment replication leads to increased network congestion on primary shards. See [Issue - Optimize network bandwidth on primary shards](https://github.com/opensearch-project/OpenSearch/issues/4245).
|
||||
1. Integration with remote-backed storage as the source of replication is [currently not supported](https://github.com/opensearch-project/OpenSearch/issues/4448).
|
||||
|
|
@ -81,7 +80,7 @@ When using segment replication, consider the following:
|
|||
|
||||
During initial benchmarks, segment replication users reported 40% higher throughput than when using document replication with the same cluster setup.
|
||||
|
||||
The following benchmarks were collected with [OpenSearch-benchmark](https://github.com/opensearch-project/opensearch-benchmark) using the [`stackoverflow`](https://www.kaggle.com/datasets/stackoverflow/stackoverflow) and [`nyc_taxi`](https://github.com/topics/nyc-taxi-dataset) datasets.
|
||||
The following benchmarks were collected with [OpenSearch-benchmark]({{site.url}}{{site.baseurl}}/benchmark/index/) using the [`stackoverflow`](https://www.kaggle.com/datasets/stackoverflow/stackoverflow) and [`nyc_taxi`](https://github.com/topics/nyc-taxi-dataset) datasets.
|
||||
|
||||
The benchmarks demonstrate the effect of the following configurations on segment replication:
|
||||
|
||||
|
|
@ -364,4 +363,4 @@ The benchmarking results show a non-zero error rate as the number of replicas in
|
|||
## Next steps
|
||||
|
||||
1. Track [future enhancements to segment replication](https://github.com/orgs/opensearch-project/projects/99).
|
||||
1. Read [this blog post about segment replication](https://opensearch.org/blog).
|
||||
1. Read [this blog post about segment replication](https://opensearch.org/blog/segment-replication/).
|
||||
Loading…
Reference in New Issue