- Deprecate the request parameter `master_timeout` that used in Index APIs which have got the parameter.
(except index template APIs, which is addressed in PR https://github.com/opensearch-project/OpenSearch/pull/2678)
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Shard level indexing pressure improves the current Indexing Pressure framework which performs memory accounting at node level and rejects the requests. This takes a step further to have rejections based on the memory accounting at shard level along with other key performance factors like throughput and last successful requests.
**Key features**
- Granular tracking of indexing tasks performance, at every shard level, for each node role i.e. coordinator, primary and replica.
- Smarter rejections by discarding the requests intended only for problematic index or shard, while still allowing others to continue (fairness in rejection).
- Rejections thresholds governed by combination of configurable parameters (such as memory limits on node) and dynamic parameters (such as latency increase, throughput degradation).
- Node level and shard level indexing pressure statistics exposed through stats api.
- Integration of Indexing pressure stats with Plugins for for metric visibility and auto-tuning in future.
- Control knobs to tune to the key performance thresholds which control rejections, to address any specific requirement or issues.
- Control knobs to run the feature in shadow-mode or enforced-mode. In shadow-mode only internal rejection breakdown metrics will be published while no actual rejections will be performed.
The changes were divided into small manageable chunks as part of the following PRs against a feature branch.
- Add Shard Indexing Pressure Settings. #716
- Add Shard Indexing Pressure Tracker. #717
- Refactor IndexingPressure to allow extension. #718
- Add Shard Indexing Pressure Store #838
- Add Shard Indexing Pressure Memory Manager #945
- Add ShardIndexingPressure framework level construct and Stats #1015
- Add Indexing Pressure Service which acts as orchestrator for IP #1084
- Add plumbing logic for IndexingPressureService in Transport Actions. #1113
- Add shard indexing pressure metric/stats via rest end point. #1171
- Add shard indexing pressure integration tests. #1198
Signed-off-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the remaining o.e.index and o.e.test packages in the
test/fixtures module. References throughout the codebase are also refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Rename directory elasticsearch to opensearch
Rename EvilElasticsearchCliTests EvilOpenSearchCliTests
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename org.elasticsearch to org.opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename waitForElasticsearch to waitForOpenSearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename OpensearchNode to OpenSearchNode
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename "elasticsearch.version" to "opensearch.version"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearchVersionString to opensearchVersionString
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.yml to opensearch.yml
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename runElasticsearchTests to runOpenSearchTests
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearchVersion to opensearchVersion
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch in gradle files
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ElasticsearchAssertions to OpenSearchAssertions
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename folder share/elasticsearch to share/opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch-service-x64 to opensearch-service-x64
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch-service.bat to opensearch-service.bat
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename Elasticsearch to Opensearch
Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ELASTIC_PASSWORD_FILE to OPENSEARCH_PASSWORD_FILE
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ELASTICSEARCH_PASSWORD to OPENSEARCH_PASSWORD
Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.log to opensearch.log
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename es-repo to opensearch-repo
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ESTestCase to OpenSearchTestCase
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ESRestTestCase to OpenSearchRestTestCase
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch
Rename "Starts ElasticSearch" to "Starts OpenSearch"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ESElasticsearchCliTestCase to BaseOpenSearchCliTestCase
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename "elasticsearch:test" to "opensearch:test"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename test91ElasticsearchShardCliPackaging to test91OpenSearchShardCliPackaging
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.toString to opensearch.toString
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.pid to opensearch.pid
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename "Opensearch" to "OpenSearch"
Rename "elasticsearch" to "opensearch"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename Elasticsearch to OpenSearch
Remove unecessary dot after opensearch.
Signed-off-by: Harold Wang <harowang@amazon.com>
