* Remove reference to an AWS instance type.
Signed-off-by: dblock <dblock@amazon.com>
* Move auto-TOC to top of doc.
Signed-off-by: dblock <dblock@amazon.com>
* Version checks are incorrectly returning versions < 1.0.0.
Signed-off-by: dblock <dblock@amazon.com>
* Removed V_7_10_3 which has not been released as of time of the fork.
Signed-off-by: dblock <dblock@amazon.com>
* Update check for current version to get unreleased versions.
- no unreleased version if the current version is "1.0.0"
- add unit tests for OpenSearch 1.0.0 with legacy ES versions.
- update VersionUtils to include all legacy ES versions as released.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Shweta Thareja <tharejas@amazon.com>
This commit changes MainResponse to spoof OpenSearch 1.x version numbers as
Legacy version number 7.10.2 for legacy clients.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* An allocation constraint mechanism, that de-prioritizes nodes from getting picked for allocation if they breach certain constraints
Signed-off-by: Ashwin Pankaj <appankaj@amazon.com>
* Update commons-io-2.4.jar to 2.7 for plugins/discovery-azure-classic module
* Remove unused jackson dependency and respective LICENSE and NOTICE
* Update guava dependency to mitigate CVE for repository-azure plugin
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
Excludes modules currently failing from missingJavadoc check.
These modules can be removed from exclusion list once javadocs are added.
Currently, only :client:rest module is enabled for missingJavadoc check.
Issue #221 tracks javadocs for legacy code.
These checks are enabled as part of gradle check. Enabling it as part of
precommit needs more work and will be done later.
Once this PR is merged all incoming PRs will undergo javadoc validations.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
We need to install elasticsearch in the test cluster nodes to be able to run the BWC tests. This commit makes the following changes to enable that,
- updates DistributionDownloadPlugin to download elasticsearch-oss distributions.
- updates OpenSearchNode to be able to run both OpenSearch and Elasticsearch distributions.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
A recent change as part of the commit c2e816ec introduced a bug where the build is failing on Windows. The change was made to include the NOTICE.txt file as read-only in the distributions. The code fails on Windows as it's not a POSIX-compliant. This commit adds a check on the current operating system.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Create group settings with fallback.
Signed-off-by: dblock <dblock@amazon.com>
* Use protected fallbackSetting in Setting.
Signed-off-by: dblock <dblock@amazon.com>
* Adds a gradle plugin to validate missing javadocs
Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fix Javadoc errors in module `client/rest` (#685)
* Fix Javadoc errors in client/rest module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add package info file in client/rest module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Fix typos
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add exception documentation to Javadoc
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Fixes precommit task configuration failures due to newly added missin… (#707)
* Fixes precommit task configuration failures due to newly added missingJavadoc task
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fixes javadoc task errors due to PR#685
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Updated CONTRIBUTING.md for info on javadocs
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Correcting licenses and naming
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Correcting version info
Signed-off-by: Himanshu Setia <setiah@amazon.com>
Co-authored-by: Gregor Zurowski <gregor@zurowski.org>
This commit adds support for data streams by adding a DataStreamFieldMapper, and making timestamp
field name configurable. Backwards compatibility is supported.
Signed-off-by: Ketan Verma <ketan9495@gmail.com>
Changes to fix the failing OpenSearch distribution tests for packages (linux-archive, linux-archive-aarch64, debian, rpm, docker) on supported linux distros.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Debian packages are formatted with the following filename structure:
name_[epoch:]version-release_arch.deb
Make generated Debian packages follow this convention.
Signed-off-by: Neal Gompa <ngompa13@gmail.com>
Hadoop 2.8.5 has been reported to have CVEs (https://bugzilla.redhat.com/show_bug.cgi?id=1883549). We need to upgrade this to 2.10.1. This also updates the hadoop-minicluster version to 2.10.1 as well. This upgrade also brings in two additional dependencies, woodstox-core and stax2-api that are added along with the sha1s, licenses and notices.
Also upgrade guava to the latest as per the CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
For discovery-gce and repository-gcs plugins update the google-oauth-client library to version 1.31.0. See CVE details at https://nvd.nist.gov/vuln/detail/CVE-2020-7692
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Add clear instructions on how to run OpenSearch with debugging mode in IntelliJ.
Fixed a few minor typos and grammars.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This change extends the remote reindex SPI to allow adding a custom interceptor.
This interceptor can be plugged in to perform any processing on the request or response.
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
* Make default number of shards configurable
The default number of primary shards for a new index, when the number of shards are not provided in the request, can be configured for the cluster.
Signed-off-by: Arunabh Singh <arunabs@amazon.com>
* Address PR comments
Signed-off-by: Arunabh Singh <arunabs@amazon.com>
Co-authored-by: Arunabh Singh <arunabs@amazon.com>
The `hadoop-minicluster:2.8.5` which is used for integration tests, has dependencies which have been flagged for having security vulnerabilities. Update this to the latest version `3.3.0`
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Changes the behavior of the recursive deletion function `executeOneStaleIndexDelete()` stop
condition to be when the queue of `staleIndicesToDelete` is empty -- also in the error flow.
Otherwise the GroupedActionListener never responds and in the event of a few exceptions the
deletion task gets stuck.
Alters the test case to fail to delete in bulk many snapshots at the first attempt, and then
the next successful deletion also takes care of the previously failed attempt as the test
originally intended.
SNAPSHOT threadpool is at most 5. So in the event we get more than 5 exceptions there are no
more threads to handle the deletion task and there is still one more snapshot to delete in the
queue. Thus, in the test I made the number of extra snapshots be one more than the max in the
SNAPSHOT threadpool.
Signed-off-by: AmiStrn <amitai.stern@logz.io>
* Add detail on how to install Java.
Signed-off-by: dblock <dblock@amazon.com>
* There's no password requirement for the instance.
Signed-off-by: dblock <dblock@amazon.com>
* Explain how to listen on an external IP.
Signed-off-by: dblock <dblock@amazon.com>
Painless tests would previously create script engine for every single test method.
Now the tests that need to tweak script engine settings create a class
level fixture (BeforeClass/AfterClass) that is used across all the test
methods in that suite.
RegexLimitTests was split into two suites (limit=1 and limit=2) rather
than dynamically applying different settings.
C2 compiler is no longer needed for tests to be fast, instead tests run
faster with C1 only as expected, like the rest of the unit tests.
Signed-off-by: Robert Muir <rmuir@apache.org>
Instead of snapshot delete of stale indices being a single threaded operation this commit makes
it a multithreaded operation and delete multiple stale indices in parallel using SNAPSHOT
threadpool's workers.
Signed-off-by: Piyush Daftary <piyush.besu@gmail.com>
Daniel Doubrovkine (dB.)
Marc Handalian
Rabi Panda
Sarat Vemulapalli
Nick Knize
Chloe
ashwinpankaj
Abbas Hussain
Neal Gompa (ニール・ゴンパ)
Himanshu Setia
Mohit Godwani
Dhwanil Patel
Ankit Jain
CEHENKLE
itiyamas
Harmish
Ketan Verma
Vacha
Sooraj Sinha
arunabh23
amitai stern
Dawn Foster
Meet Shah
Robert Muir
piyush