* An allocation constraint mechanism, that de-prioritizes nodes from getting picked for allocation if they breach certain constraints
Signed-off-by: Ashwin Pankaj <appankaj@amazon.com>
* Update commons-io-2.4.jar to 2.7 for plugins/discovery-azure-classic module
* Remove unused jackson dependency and respective LICENSE and NOTICE
* Update guava dependency to mitigate CVE for repository-azure plugin
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
Excludes modules currently failing from missingJavadoc check.
These modules can be removed from exclusion list once javadocs are added.
Currently, only :client:rest module is enabled for missingJavadoc check.
Issue #221 tracks javadocs for legacy code.
These checks are enabled as part of gradle check. Enabling it as part of
precommit needs more work and will be done later.
Once this PR is merged all incoming PRs will undergo javadoc validations.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
We need to install elasticsearch in the test cluster nodes to be able to run the BWC tests. This commit makes the following changes to enable that,
- updates DistributionDownloadPlugin to download elasticsearch-oss distributions.
- updates OpenSearchNode to be able to run both OpenSearch and Elasticsearch distributions.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
A recent change as part of the commit c2e816ec introduced a bug where the build is failing on Windows. The change was made to include the NOTICE.txt file as read-only in the distributions. The code fails on Windows as it's not a POSIX-compliant. This commit adds a check on the current operating system.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Create group settings with fallback.
Signed-off-by: dblock <dblock@amazon.com>
* Use protected fallbackSetting in Setting.
Signed-off-by: dblock <dblock@amazon.com>
* Adds a gradle plugin to validate missing javadocs
Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fix Javadoc errors in module `client/rest` (#685)
* Fix Javadoc errors in client/rest module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add package info file in client/rest module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Fix typos
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add exception documentation to Javadoc
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Fixes precommit task configuration failures due to newly added missin… (#707)
* Fixes precommit task configuration failures due to newly added missingJavadoc task
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fixes javadoc task errors due to PR#685
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Updated CONTRIBUTING.md for info on javadocs
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Correcting licenses and naming
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Correcting version info
Signed-off-by: Himanshu Setia <setiah@amazon.com>
Co-authored-by: Gregor Zurowski <gregor@zurowski.org>
This commit adds support for data streams by adding a DataStreamFieldMapper, and making timestamp
field name configurable. Backwards compatibility is supported.
Signed-off-by: Ketan Verma <ketan9495@gmail.com>
Changes to fix the failing OpenSearch distribution tests for packages (linux-archive, linux-archive-aarch64, debian, rpm, docker) on supported linux distros.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Debian packages are formatted with the following filename structure:
name_[epoch:]version-release_arch.deb
Make generated Debian packages follow this convention.
Signed-off-by: Neal Gompa <ngompa13@gmail.com>
Hadoop 2.8.5 has been reported to have CVEs (https://bugzilla.redhat.com/show_bug.cgi?id=1883549). We need to upgrade this to 2.10.1. This also updates the hadoop-minicluster version to 2.10.1 as well. This upgrade also brings in two additional dependencies, woodstox-core and stax2-api that are added along with the sha1s, licenses and notices.
Also upgrade guava to the latest as per the CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
For discovery-gce and repository-gcs plugins update the google-oauth-client library to version 1.31.0. See CVE details at https://nvd.nist.gov/vuln/detail/CVE-2020-7692
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Add clear instructions on how to run OpenSearch with debugging mode in IntelliJ.
Fixed a few minor typos and grammars.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This change extends the remote reindex SPI to allow adding a custom interceptor.
This interceptor can be plugged in to perform any processing on the request or response.
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
* Make default number of shards configurable
The default number of primary shards for a new index, when the number of shards are not provided in the request, can be configured for the cluster.
Signed-off-by: Arunabh Singh <arunabs@amazon.com>
* Address PR comments
Signed-off-by: Arunabh Singh <arunabs@amazon.com>
Co-authored-by: Arunabh Singh <arunabs@amazon.com>
The `hadoop-minicluster:2.8.5` which is used for integration tests, has dependencies which have been flagged for having security vulnerabilities. Update this to the latest version `3.3.0`
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Changes the behavior of the recursive deletion function `executeOneStaleIndexDelete()` stop
condition to be when the queue of `staleIndicesToDelete` is empty -- also in the error flow.
Otherwise the GroupedActionListener never responds and in the event of a few exceptions the
deletion task gets stuck.
Alters the test case to fail to delete in bulk many snapshots at the first attempt, and then
the next successful deletion also takes care of the previously failed attempt as the test
originally intended.
SNAPSHOT threadpool is at most 5. So in the event we get more than 5 exceptions there are no
more threads to handle the deletion task and there is still one more snapshot to delete in the
queue. Thus, in the test I made the number of extra snapshots be one more than the max in the
SNAPSHOT threadpool.
Signed-off-by: AmiStrn <amitai.stern@logz.io>
* Add detail on how to install Java.
Signed-off-by: dblock <dblock@amazon.com>
* There's no password requirement for the instance.
Signed-off-by: dblock <dblock@amazon.com>
* Explain how to listen on an external IP.
Signed-off-by: dblock <dblock@amazon.com>
Painless tests would previously create script engine for every single test method.
Now the tests that need to tweak script engine settings create a class
level fixture (BeforeClass/AfterClass) that is used across all the test
methods in that suite.
RegexLimitTests was split into two suites (limit=1 and limit=2) rather
than dynamically applying different settings.
C2 compiler is no longer needed for tests to be fast, instead tests run
faster with C1 only as expected, like the rest of the unit tests.
Signed-off-by: Robert Muir <rmuir@apache.org>
Instead of snapshot delete of stale indices being a single threaded operation this commit makes
it a multithreaded operation and delete multiple stale indices in parallel using SNAPSHOT
threadpool's workers.
Signed-off-by: Piyush Daftary <piyush.besu@gmail.com>
The change in 0ba0e7cc, introduced the issue where randomly selecting an incompatible version fails the test. It caused the filtering logic to incorrectly identify all ES 7.*.* versions as bad versions for joining which should not be the case.
Additionally, split the test into two separate tests where earlier only one of them was run at random.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes the Version.fromString logic to identify legacy versions. It
also adds an optional "distribution" field to the MainRespose for OpenSearch
version 1.0.0+. Any preceeding versions that do not contain the distribution
label will be handeled as legacy versions appropriately.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit removes the 'oss' string which was a remnant of the predecessor distribution flavors. As OpenSearch has no flavors for distributions, we are removing this tag from all the distribution names.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
ashwinpankaj
Abbas Hussain
Neal Gompa (ニール・ゴンパ)
Himanshu Setia
Rabi Panda
Mohit Godwani
Daniel Doubrovkine (dB.)
Dhwanil Patel
Ankit Jain
CEHENKLE
itiyamas
Harmish
Ketan Verma
Sarat Vemulapalli
Vacha
Sooraj Sinha
arunabh23
amitai stern
Dawn Foster
Meet Shah
Robert Muir
piyush
Nick Knize
Tom "spot" Callaway