* Made changes.
Signed-off-by: Megha Sai Kavikondala <kavmegha@amazon.com>
* Signed-off-by: Megha Sai Kavikondala <kavmegha@amazon.com>
Changes made by deleting the TestSettingsIT file and adding new lines in FullClusterRestartSettingsUpgradeIT.java
* Added logic to allow {dot} files on startup
Signed-off-by: Ryan Bogan <rbogan@amazon.com>
* Ensures that only plugin directories are returned by findPluginDirs()
Signed-off-by: Ryan Bogan <rbogan@amazon.com>
* Prevents . files from being returned as plugins
Signed-off-by: Ryan Bogan <rbogan@amazon.com>
* Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Using -Djava.security.egd=file:/dev/urandom explicitly for cli tests
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Shard level indexing pressure improves the current Indexing Pressure framework which performs memory accounting at node level and rejects the requests. This takes a step further to have rejections based on the memory accounting at shard level along with other key performance factors like throughput and last successful requests.
**Key features**
- Granular tracking of indexing tasks performance, at every shard level, for each node role i.e. coordinator, primary and replica.
- Smarter rejections by discarding the requests intended only for problematic index or shard, while still allowing others to continue (fairness in rejection).
- Rejections thresholds governed by combination of configurable parameters (such as memory limits on node) and dynamic parameters (such as latency increase, throughput degradation).
- Node level and shard level indexing pressure statistics exposed through stats api.
- Integration of Indexing pressure stats with Plugins for for metric visibility and auto-tuning in future.
- Control knobs to tune to the key performance thresholds which control rejections, to address any specific requirement or issues.
- Control knobs to run the feature in shadow-mode or enforced-mode. In shadow-mode only internal rejection breakdown metrics will be published while no actual rejections will be performed.
The changes were divided into small manageable chunks as part of the following PRs against a feature branch.
- Add Shard Indexing Pressure Settings. #716
- Add Shard Indexing Pressure Tracker. #717
- Refactor IndexingPressure to allow extension. #718
- Add Shard Indexing Pressure Store #838
- Add Shard Indexing Pressure Memory Manager #945
- Add ShardIndexingPressure framework level construct and Stats #1015
- Add Indexing Pressure Service which acts as orchestrator for IP #1084
- Add plumbing logic for IndexingPressureService in Transport Actions. #1113
- Add shard indexing pressure metric/stats via rest end point. #1171
- Add shard indexing pressure integration tests. #1198
Signed-off-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Shweta Thareja <tharejas@amazon.com>
Changes to fix the failing OpenSearch distribution tests for packages (linux-archive, linux-archive-aarch64, debian, rpm, docker) on supported linux distros.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Make default number of shards configurable
The default number of primary shards for a new index, when the number of shards are not provided in the request, can be configured for the cluster.
Signed-off-by: Arunabh Singh <arunabs@amazon.com>
* Address PR comments
Signed-off-by: Arunabh Singh <arunabs@amazon.com>
Co-authored-by: Arunabh Singh <arunabs@amazon.com>
This commit removes the 'oss' string which was a remnant of the predecessor distribution flavors. As OpenSearch has no flavors for distributions, we are removing this tag from all the distribution names.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit rebases the versioning to OpenSearch 1.0.0
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
While creating the archives and packages, the build tries to copy the non-existent file `README.asciidoc` instead of `README.md`. Consequently, the packaging tests fail during verification time. This commit addresses the issue by fixing the name.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit adds the SPDX license header and modifications copyright to security
policy files.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit removes the references for the `default` docker distributions which were originally part of the different flavors of distributions. This also fixes some of the failing docker compose tests under `qa`.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes some more renaming issues and as a result fixes the failing tests,
* :qa:logging-config:test
* :example-plugins:painless-whitelist:yamlRestTest
* :modules:reindex:test
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes some name issues leftover from the rename to OpenSearch work.
With this commit, the `gradlew :run` task should work.
Signed-off-by: Rabi Panda <pandarab@amazon.com>
This commit refactors instances of 'elasticsearch' with opensearch everywhere
except references to issues, and other places needed to test compatibility with
old elasticsearch clusters.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Fix miscellaneous issues identified during `gradle precommit`. These issues are the side effects of the renaming to OpenSearch work.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors remaining ES classes to OpenSearch prefix throughout the
code base. All references are also refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit fixes the currently broken gradle build resulted from the renaming work. It reverts a few dependencies and comments out the `opensearch_distibutions` task which is currently failing for some builds. We will address these separately in the future once we have a working build.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit cleans up imports, variable names, comments, and other misc usages
of ES with the new OpenSearch name.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the remaining o.e.index and o.e.test packages in the
test/fixtures module. References throughout the codebase are also refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the heavily used ESPolicy, Elasticsearch (main class), and Elasticsearch
prefixed test classes used in the bootstrap package under the server module. Refactoring the
namespace will come in a separate commit.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
* Rename directory elasticsearch to opensearch
Rename EvilElasticsearchCliTests EvilOpenSearchCliTests
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename org.elasticsearch to org.opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename waitForElasticsearch to waitForOpenSearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename OpensearchNode to OpenSearchNode
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename "elasticsearch.version" to "opensearch.version"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearchVersionString to opensearchVersionString
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.yml to opensearch.yml
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename runElasticsearchTests to runOpenSearchTests
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearchVersion to opensearchVersion
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch in gradle files
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ElasticsearchAssertions to OpenSearchAssertions
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename folder share/elasticsearch to share/opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch-service-x64 to opensearch-service-x64
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch-service.bat to opensearch-service.bat
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename Elasticsearch to Opensearch
Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ELASTIC_PASSWORD_FILE to OPENSEARCH_PASSWORD_FILE
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ELASTICSEARCH_PASSWORD to OPENSEARCH_PASSWORD
Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.log to opensearch.log
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename es-repo to opensearch-repo
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ESTestCase to OpenSearchTestCase
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ESRestTestCase to OpenSearchRestTestCase
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch to opensearch
Rename "Starts ElasticSearch" to "Starts OpenSearch"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename ESElasticsearchCliTestCase to BaseOpenSearchCliTestCase
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename "elasticsearch:test" to "opensearch:test"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename test91ElasticsearchShardCliPackaging to test91OpenSearchShardCliPackaging
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.toString to opensearch.toString
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename elasticsearch.pid to opensearch.pid
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename "Opensearch" to "OpenSearch"
Rename "elasticsearch" to "opensearch"
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename Elasticsearch to OpenSearch
Remove unecessary dot after opensearch.
Signed-off-by: Harold Wang <harowang@amazon.com>
* [Rename] Rename qa folder
Signed-off-by: Harold Wang <harowang@amazon.com>
* Remove the dot in the end of "package org.opensearch."
Signed-off-by: Harold Wang <harowang@amazon.com>
* Add semicolon
Signed-off-by: Harold Wang <harowang@amazon.com>
This commit refactors the ElasticsearchStatusException in the server module to
OpenSearchStatusException. References and usages throughout the rest of the
codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the ElasticsearchException class located in the server module
to OpenSearchException. References and usages throughout the rest of the
codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit changes the building, packaging, and testing framework to only support OSS on different distributions.
Next steps:
completely remove -oss flag dependencies in package and build tests
move 6.x bwc testing to be an explicit option
remove any references to elastic.co download site (or replace with downloads from the OSS website)
Co-authored-by: Himanshu Setia <setiah@amazon.com>
Co-authored-by: Rabi Panda <pandarab@amazon.com>
Co-authored-by: Himanshu Setia <58999915+setiah@users.noreply.github.com>
Co-authored-by: Sarat Vemulapalli <vemsarat@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
Andriy Redko
Ryan Bogan
Megha Sai Kavikondala
Himanshu Setia
Vacha
Saurabh Singh
Nick Knize
Sven R
Daniel Doubrovkine (dB.)
shwetathareja
Vlad Rozov
Nick Knize
Rabi Panda
arunabh23
Tianli Feng
Rabi Panda
Harold Wang
Nicholas Knize