This change updates the logic in RunTask to use deterministic port values for all nodes rather than just the first node in a cluster. The local address of the first node is also used as the seed_hosts value to allow the cluster to form correctly. The seed_hosts setting is also allow-listed as an overrideable value.
Signed-off-by: Kartik Ganesh <85275476+kartg@users.noreply.github.com>
As part of the commit 2ebd0e04, we added a new method to the EnginePlugin to provide a
custom TranslogDeletionPolicy. This commit makes minTranslogGenRequired method
abstract in this class for implementation by child classes. The default implementation
is provided by DefaultTranslogDeletionPolicy.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
The settings and the corresponding logic for translog pruning by retention lease which were added as part of #1100 have been deprecated. This commit removes those deprecated code in favor of an extension point for providing a custom TranslogDeletionPolicy.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
When creating the second instance of an InternalEngine using the same
translog config of the default InternalEngine instance, the second
instance will attempt to delete all the existing translog files. I found
a deterministic test failure when running with the seed
`E3E6AAD95ABD299B`.
As opposed to creating a second engine instance with a different
translog location, just close the first one before creating the second.
Signed-off-by: Andrew Ross <andrross@amazon.com>
This commit adds a method that can be used to provide a custom TranslogDeletionPolicy
from within plugins that implement the EnginePlugin interface. This enables plugins to
provide a custom deletion policy with the current limitation that only one plugin can
override the policy. An exception will be thrown if more than one plugin overrides the
policy.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit adds an explicit version check for shard indexing pressure
serialization. This is required to not mandate test have the cluster service
initialized while asserting node attributes.
Signed-off-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Saurabh Singh <sisurab@amazon.com>
This commit adds an extension point to EngineConfig through EnginePlugin using
a new EngineConfigFactory mechanism. EnginePlugin provides interface methods to
override configurations in EngineConfig. The EngineConfigFactory produces a new
instance of the EngineConfig using these overrides. Defaults are used absent
overridden configurations.
This serves as a mechanism to override Engine configurations (e.g., CodecService,
TranslogConfig) enabling Plugins to have higher fidelity for changing Engine
behavior without having to override the entire Engine (which is only permitted for
a single plugin).
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Modernize and consolidate JDKs usage across all stages of the build. Update JDK-14 requirement, switch to JDK-17 instead
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Updating phrasing based on review feedback
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Fixed runtime Java version usage, it has to be respected when RUNTIME_JAVA_HOME == JAVA_HOME
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Addressing review comments
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Fixes flakiness for test testReplicaThreadedThroughputDegradationAndRejection.
Reduced the number of threads concurrently executing from the initial range between (100-120) to a new range between (80-100), as the previous range was breaking the node limits set as 10kb, for every execution where number of threads were greater than 110.
Signed-off-by: Saurabh Singh <sisurab@amazon.com>
* Allow building on FreeBSD
With this set of change, we are able to successfuly run:
```
./gradlew publishToMavenLocal -Dbuild.snapshot=false
```
This step is used in the OpenSearch repository context when building
plugins in the current state of the CI.
While here, reorder OS conditions alphabetically.
Before building, the openjdk14 package was installed and the environment
was adjusted to use it:
```
sudo pkg install openjdk14
export JAVA_HOME=/usr/local/openjdk14/
export PATH=$JAVA_HOME/bin:$PATH
```
Signed-off-by: Romain Tartière <romain@blogreen.org>
* Unbreak CI with FreeBSD support
Signed-off-by: dblock <dblock@dblock.org>
Co-authored-by: dblock <dblock@dblock.org>
* Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Using -Djava.security.egd=file:/dev/urandom explicitly for cli tests
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Shard level indexing pressure improves the current Indexing Pressure framework which performs memory accounting at node level and rejects the requests. This takes a step further to have rejections based on the memory accounting at shard level along with other key performance factors like throughput and last successful requests.
**Key features**
- Granular tracking of indexing tasks performance, at every shard level, for each node role i.e. coordinator, primary and replica.
- Smarter rejections by discarding the requests intended only for problematic index or shard, while still allowing others to continue (fairness in rejection).
- Rejections thresholds governed by combination of configurable parameters (such as memory limits on node) and dynamic parameters (such as latency increase, throughput degradation).
- Node level and shard level indexing pressure statistics exposed through stats api.
- Integration of Indexing pressure stats with Plugins for for metric visibility and auto-tuning in future.
- Control knobs to tune to the key performance thresholds which control rejections, to address any specific requirement or issues.
- Control knobs to run the feature in shadow-mode or enforced-mode. In shadow-mode only internal rejection breakdown metrics will be published while no actual rejections will be performed.
The changes were divided into small manageable chunks as part of the following PRs against a feature branch.
- Add Shard Indexing Pressure Settings. #716
- Add Shard Indexing Pressure Tracker. #717
- Refactor IndexingPressure to allow extension. #718
- Add Shard Indexing Pressure Store #838
- Add Shard Indexing Pressure Memory Manager #945
- Add ShardIndexingPressure framework level construct and Stats #1015
- Add Indexing Pressure Service which acts as orchestrator for IP #1084
- Add plumbing logic for IndexingPressureService in Transport Actions. #1113
- Add shard indexing pressure metric/stats via rest end point. #1171
- Add shard indexing pressure integration tests. #1198
Signed-off-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
* Support for Heap after GC stats
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Addressing code review comments
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Using the right version 2.0.0 (instead of 1.2.0) for the change
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
This commit adds test coverage for translog pruning setting to
MetadataCreateIndexServiceTests
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Rename translog pruing setting to CCR specific setting
Signed-off-by: Sai Kumar <karanas@amazon.com>
* Rename to index.plugins.replication.translog.retention_lease.pruning.enabled as
index settings needs "index." as prefix
Signed-off-by: Sai Kumar <karanas@amazon.com>
* Add deprecations to retention pruning controls
This commit adds deprecation flags to all added settings, variables, and methods
specific to ccr's retention lease pruning mechanism.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Addressed CR comments
Signed-off-by: Sai Kumar <karanas@amazon.com>
* fix javadoc deprecation
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* fix deprecation tag in TranslogDeletionPolicy
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Addressed test issue under translog tests
Signed-off-by: Sai Kumar <karanas@amazon.com>
Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
kartg
Andriy Redko
Rabi Panda
Sarat Vemulapalli
Owais Kazi
Nick Knize
CEHENKLE
Andrew Ross
Vacha
Kyle J. Davis
Daniel Doubrovkine (dB.)
Saurabh Singh
Xue Zhou
Sergey Nuyanzin
Romain Tartière
Rishikesh Pasham
Shivansh Arora
Sai