with Elasticsearch there were two flavours of the docker build: one for
the OSS build and one with X-Pack. accordingly, there were two
`log4j2.properties` files for the two cases and the `build.gradle`
picked up the correct one depending on the build.
commit 83e87f7e54 removed all non-OSS functionality from the build but
with that it also removed the logic which so far copied over the OSS
version of `log4j2.properties`.
what the diff doesn't show properly here is that `log4j2.properties` has
been deleted and `oss/log4j2.properties` has been moved into its place
without any changes to it.
this fixes#868
Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>
This change adds a new cluster setting "compatibility.override_main_response_version"
that when enabled spoofs the version.number returned from MainResponse
for REST clients expecting legacy version 7.10.2.
Signed-off-by: Marc Handalian <handalm@amazon.com>
OpenSearch uses plugins to add new features. A plugin can be included in the distribution by default (as part of *modules* directory) or can be installed optionally from a plugin repository.
This change provides a separate space called *sandbox* inside OpenSearch for the community to easily experiment new ideas and innovate. Ideally, this is where an experimental feature will reside before it can be promoted to the *modules* directory. All the plugins in this module will only be included in the **snapshot distributions**. During assembling distributions, we will check if the following two conditions are met to include the sandbox modules,
* the distribution is a snapshot i.e. the build system property `build.snapshot` is set to true. We use this because it will prevent accidental bundling of these modules in a release distribution.
* the `enable.sandbox` system property is set to true. By default it is set to true. The purpose of adding this extra flag is that we can exclude the modules from snapshots if needed. For example, we may want to run performance tests on snapshots without the sandbox modules.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Add Plugin name for verbose Plugin not found exception
* Make the plugin loading failure exception more verbose
* Throw Opensearch in place of RuntimeException for plugin load failure
* Nit fix, added ... to make logging standout
Signed-off-by: Jayesh Hathila <sharma.jayesh52@gmail.com>
* Address a kind of issue suggested by Amazon CodeGuru Reviewer:
* Add try-with-resources block to automatically close the resources after using to avoid resource leak, in `SymbolicLinkPreservingTarIT`, `LicenseAnalyzer`, `SymbolicLinkPreservingUntarTransform`, `ConcurrentSeqNoVersioningIT` in `VersionProperties`, `GeoFilterIT`, `XContentHelper`, `Json` and `IndexShard` class
* Add try-finally block to close the resources after using to avoid resource leak, in `ServerChannelContext` class.
* Add try-catch block to close the resources when exception occurs in `FsBlobContainer` class (when XContentFactory.xContentType throws an exception).
* Close resources when assertion error occurs, in `ServerChannelContext` class.
Found when attempting to build on an `arm64` machine when I recieved an error message below. Root cause is that string equality in java cannot be done with the `==` sign.
```
unknown architecture [arm64] for jdk [provisioned_runtime], must be one of [aarch64, x64]
```
Signed-off-by: Peter Nied <petern@amazon.com>
* Adds a gradle plugin to validate missing javadocs
Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fix Javadoc errors in client/sniffer module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add package info to client/sniffer module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
Co-authored-by: Himanshu Setia <setiah@amazon.com>
* Remove reference to an AWS instance type.
Signed-off-by: dblock <dblock@amazon.com>
* Move auto-TOC to top of doc.
Signed-off-by: dblock <dblock@amazon.com>
* Version checks are incorrectly returning versions < 1.0.0.
Signed-off-by: dblock <dblock@amazon.com>
* Removed V_7_10_3 which has not been released as of time of the fork.
Signed-off-by: dblock <dblock@amazon.com>
* Update check for current version to get unreleased versions.
- no unreleased version if the current version is "1.0.0"
- add unit tests for OpenSearch 1.0.0 with legacy ES versions.
- update VersionUtils to include all legacy ES versions as released.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Shweta Thareja <tharejas@amazon.com>
This commit changes MainResponse to spoof OpenSearch 1.x version numbers as
Legacy version number 7.10.2 for legacy clients.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* An allocation constraint mechanism, that de-prioritizes nodes from getting picked for allocation if they breach certain constraints
Signed-off-by: Ashwin Pankaj <appankaj@amazon.com>
* Update commons-io-2.4.jar to 2.7 for plugins/discovery-azure-classic module
* Remove unused jackson dependency and respective LICENSE and NOTICE
* Update guava dependency to mitigate CVE for repository-azure plugin
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
Excludes modules currently failing from missingJavadoc check.
These modules can be removed from exclusion list once javadocs are added.
Currently, only :client:rest module is enabled for missingJavadoc check.
Issue #221 tracks javadocs for legacy code.
These checks are enabled as part of gradle check. Enabling it as part of
precommit needs more work and will be done later.
Once this PR is merged all incoming PRs will undergo javadoc validations.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
We need to install elasticsearch in the test cluster nodes to be able to run the BWC tests. This commit makes the following changes to enable that,
- updates DistributionDownloadPlugin to download elasticsearch-oss distributions.
- updates OpenSearchNode to be able to run both OpenSearch and Elasticsearch distributions.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
A recent change as part of the commit c2e816ec introduced a bug where the build is failing on Windows. The change was made to include the NOTICE.txt file as read-only in the distributions. The code fails on Windows as it's not a POSIX-compliant. This commit adds a check on the current operating system.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Create group settings with fallback.
Signed-off-by: dblock <dblock@amazon.com>
* Use protected fallbackSetting in Setting.
Signed-off-by: dblock <dblock@amazon.com>
* Adds a gradle plugin to validate missing javadocs
Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fix Javadoc errors in module `client/rest` (#685)
* Fix Javadoc errors in client/rest module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add package info file in client/rest module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Fix typos
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add exception documentation to Javadoc
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Fixes precommit task configuration failures due to newly added missin… (#707)
* Fixes precommit task configuration failures due to newly added missingJavadoc task
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fixes javadoc task errors due to PR#685
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Updated CONTRIBUTING.md for info on javadocs
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Correcting licenses and naming
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Correcting version info
Signed-off-by: Himanshu Setia <setiah@amazon.com>
Co-authored-by: Gregor Zurowski <gregor@zurowski.org>
This commit adds support for data streams by adding a DataStreamFieldMapper, and making timestamp
field name configurable. Backwards compatibility is supported.
Signed-off-by: Ketan Verma <ketan9495@gmail.com>
Changes to fix the failing OpenSearch distribution tests for packages (linux-archive, linux-archive-aarch64, debian, rpm, docker) on supported linux distros.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Debian packages are formatted with the following filename structure:
name_[epoch:]version-release_arch.deb
Make generated Debian packages follow this convention.
Signed-off-by: Neal Gompa <ngompa13@gmail.com>
Hadoop 2.8.5 has been reported to have CVEs (https://bugzilla.redhat.com/show_bug.cgi?id=1883549). We need to upgrade this to 2.10.1. This also updates the hadoop-minicluster version to 2.10.1 as well. This upgrade also brings in two additional dependencies, woodstox-core and stax2-api that are added along with the sha1s, licenses and notices.
Also upgrade guava to the latest as per the CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
For discovery-gce and repository-gcs plugins update the google-oauth-client library to version 1.31.0. See CVE details at https://nvd.nist.gov/vuln/detail/CVE-2020-7692
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Add clear instructions on how to run OpenSearch with debugging mode in IntelliJ.
Fixed a few minor typos and grammars.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Nick Knize
Tianli Feng
Ralph Ursprung
Marc Handalian
Rabi Panda
Daniel Doubrovkine (dB.)
itiyamas
jayesh hathila
Jesper Olsson
Gregor Zurowski
Peter Nied
Sarat Vemulapalli
Chloe
ashwinpankaj
Abbas Hussain
Neal Gompa (ニール・ゴンパ)
Himanshu Setia
Mohit Godwani
Dhwanil Patel
Ankit Jain
CEHENKLE
Harmish
Ketan Verma