Currently, the permissions for opensearch logs are -rw-r-r-, which gives read access to anyone. This weak permission
structure can lead to leakage of any sensitive information (if published) in the logs. This commit restricts read
access with -rw-r--- permission.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
This change adds the initial version of a new CLI tool `opensearch-upgrade` as part of the OpenSearch distribution. This tool is meant for assisting during an upgrade from an existing Elasticsearch v7.10.2/v6.8.0 node to OpenSearch. It automates the process of importing existing configurations and installing of core plugins.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Adding broken links checker
Signed-off-by: Vacha Shah <vachshah@amazon.com>
* Adding exclusions for links
Signed-off-by: Vacha Shah <vachshah@amazon.com>
* Correcting broken link
Signed-off-by: Vacha Shah <vachshah@amazon.com>
* Removing the benchmarks link
Signed-off-by: Vacha Shah <vachshah@amazon.com>
* Lower build requirement from Java 14+ to Java 11+
Avoid use of -Werror -Xlint:all, which may change significantly across
java releases (new warnings could be added). Instead, just list the
warnings individually.
Workaround JDK 11 compiler bug (JDK-8209058) that only impacts test fixture
code in the build itself.
Signed-off-by: Robert Muir <rmuir@apache.org>
* Disable warning around -source 7 -release 7 for java version checker
The java version checker triggers some default warnings because it
targets java7:
```
> Task :distribution:tools:java-version-checker:compileJava FAILED
warning: [options] source value 7 is obsolete and will be removed in a future release
warning: [options] target value 7 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
error: warnings found and -Werror specified
```
Suppress this warning explicitly for this module.
Signed-off-by: Robert Muir <rmuir@apache.org>
* more java14 -> java11 cleanup
Signed-off-by: Robert Muir <rmuir@apache.org>
Co-authored-by: Robert Muir <rmuir@apache.org>
Fixes the cat.health yaml failures when running in a bwc mixed cluster with
legacy (pre 1.0.0) nodes.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Add "tagline" field back to "MainResponse" in sever side (not in rest-high-level-client side) that removed in PR #427 .
* Replace with a new tagline "The OpenSearch Project: https://opensearch.org/".
* Turn the tagline into a constant in server/src/main/java/org/opensearch/action/main/MainResponse.java.
Snapshots are published with a -latest flag into s3, this updates
the pattern to correctly point to -latest.
Signed-off-by: Marc Handalian <handalm@amazon.com>
This Change removes version.distribution when the version.number is
overridden with the cluster setting compatibility.override_main_response_version.
Signed-off-by: Marc Handalian <handalm@amazon.com>
This change updates DistributionDownloader by registering a second
repository containing release-candidate bundles. This repository will
only be checked if the release repository does not contain the requested
version.
This change also updates the snapshot repository to point to a new
snapshot repository.
Signed-off-by: Marc Handalian <handalm@amazon.com>
with Elasticsearch there were two flavours of the docker build: one for
the OSS build and one with X-Pack. accordingly, there were two
`log4j2.properties` files for the two cases and the `build.gradle`
picked up the correct one depending on the build.
commit 83e87f7e54 removed all non-OSS functionality from the build but
with that it also removed the logic which so far copied over the OSS
version of `log4j2.properties`.
what the diff doesn't show properly here is that `log4j2.properties` has
been deleted and `oss/log4j2.properties` has been moved into its place
without any changes to it.
this fixes#868
Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>
This change adds a new cluster setting "compatibility.override_main_response_version"
that when enabled spoofs the version.number returned from MainResponse
for REST clients expecting legacy version 7.10.2.
Signed-off-by: Marc Handalian <handalm@amazon.com>
OpenSearch uses plugins to add new features. A plugin can be included in the distribution by default (as part of *modules* directory) or can be installed optionally from a plugin repository.
This change provides a separate space called *sandbox* inside OpenSearch for the community to easily experiment new ideas and innovate. Ideally, this is where an experimental feature will reside before it can be promoted to the *modules* directory. All the plugins in this module will only be included in the **snapshot distributions**. During assembling distributions, we will check if the following two conditions are met to include the sandbox modules,
* the distribution is a snapshot i.e. the build system property `build.snapshot` is set to true. We use this because it will prevent accidental bundling of these modules in a release distribution.
* the `enable.sandbox` system property is set to true. By default it is set to true. The purpose of adding this extra flag is that we can exclude the modules from snapshots if needed. For example, we may want to run performance tests on snapshots without the sandbox modules.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* Add Plugin name for verbose Plugin not found exception
* Make the plugin loading failure exception more verbose
* Throw Opensearch in place of RuntimeException for plugin load failure
* Nit fix, added ... to make logging standout
Signed-off-by: Jayesh Hathila <sharma.jayesh52@gmail.com>
* Address a kind of issue suggested by Amazon CodeGuru Reviewer:
* Add try-with-resources block to automatically close the resources after using to avoid resource leak, in `SymbolicLinkPreservingTarIT`, `LicenseAnalyzer`, `SymbolicLinkPreservingUntarTransform`, `ConcurrentSeqNoVersioningIT` in `VersionProperties`, `GeoFilterIT`, `XContentHelper`, `Json` and `IndexShard` class
* Add try-finally block to close the resources after using to avoid resource leak, in `ServerChannelContext` class.
* Add try-catch block to close the resources when exception occurs in `FsBlobContainer` class (when XContentFactory.xContentType throws an exception).
* Close resources when assertion error occurs, in `ServerChannelContext` class.
Found when attempting to build on an `arm64` machine when I recieved an error message below. Root cause is that string equality in java cannot be done with the `==` sign.
```
unknown architecture [arm64] for jdk [provisioned_runtime], must be one of [aarch64, x64]
```
Signed-off-by: Peter Nied <petern@amazon.com>
* Adds a gradle plugin to validate missing javadocs
Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.
Signed-off-by: Himanshu Setia <setiah@amazon.com>
* Fix Javadoc errors in client/sniffer module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
* Add package info to client/sniffer module
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
Co-authored-by: Himanshu Setia <setiah@amazon.com>
* Remove reference to an AWS instance type.
Signed-off-by: dblock <dblock@amazon.com>
* Move auto-TOC to top of doc.
Signed-off-by: dblock <dblock@amazon.com>
* Version checks are incorrectly returning versions < 1.0.0.
Signed-off-by: dblock <dblock@amazon.com>
* Removed V_7_10_3 which has not been released as of time of the fork.
Signed-off-by: dblock <dblock@amazon.com>
* Update check for current version to get unreleased versions.
- no unreleased version if the current version is "1.0.0"
- add unit tests for OpenSearch 1.0.0 with legacy ES versions.
- update VersionUtils to include all legacy ES versions as released.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Shweta Thareja <tharejas@amazon.com>
This commit changes MainResponse to spoof OpenSearch 1.x version numbers as
Legacy version number 7.10.2 for legacy clients.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* An allocation constraint mechanism, that de-prioritizes nodes from getting picked for allocation if they breach certain constraints
Signed-off-by: Ashwin Pankaj <appankaj@amazon.com>
* Update commons-io-2.4.jar to 2.7 for plugins/discovery-azure-classic module
* Remove unused jackson dependency and respective LICENSE and NOTICE
* Update guava dependency to mitigate CVE for repository-azure plugin
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
afazel
Vlad Rozov
Himanshu Setia
Chang Liu
Rabi Panda
Vacha
Daniel Doubrovkine (dB.)
Abbas Hussain
Sooraj Sinha
Nick Knize
Marc Handalian
Rabi Panda
shwetathareja
Tianli Feng
Ralph Ursprung
itiyamas
jayesh hathila
Jesper Olsson
Gregor Zurowski
Peter Nied
Sarat Vemulapalli
Chloe
ashwinpankaj
Neal Gompa (ニール・ゴンパ)