* Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Using -Djava.security.egd=file:/dev/urandom explicitly for cli tests
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
The public key has changed since the initial release. This commit fixes the
public key and uses the .sig files that are published to the artifacts site.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit rebases the versioning to OpenSearch 1.0.0
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit updates the public_key.asc for verifying the signature of plugins in
the InstallPluginCommand utility. The key details are as follows:
e-mail: opensearch-infra@amazon.com
key id: 0934A65836A51424
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors instances of 'elasticsearch' with opensearch everywhere
except references to issues, and other places needed to test compatibility with
old elasticsearch clusters.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Fix miscellaneous issues identified during `gradle precommit`. These issues are the side effects of the renaming to OpenSearch work.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors the remaining o.e.index and o.e.test packages in the
test/fixtures module. References throughout the codebase are also refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the following test framework packages:
* o.e.env
* o.e.geo
* o.e.http
* o.e.indices
* o.e.ingest
* o.e.plugin
* o.e.upgrades
to the o.opensearch namespace. All references throughout the test codebase have
been refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the following test packages from the o.e namespace:
* o.e.action
* o.e.bootstrap
* o.e.cli
* o.e.client
* o.e.cluster
* o.e.common
to the o.opensearch namespace. Any references throughout the codebase are also
refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the remaining classes from o.e to the o.opensearch
namespace. All references throughout the codebase have been refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Refactor the code in the `libs/core` module and any references to those in the entire code base. The refactoring is done as part of the renaming to OpenSearch work.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors o.e.common.settings package to the
o.opensearch.common.setttings namespace. All references throughout the codebase
are refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the following packages:
* o.e.common.geo
* o.e.common.hash
* o.e.common.io
into the o.opensearch.common namespace. All references throughout the codebase
have been refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
Refactor the `distribution` module to rename elasticsearch to opensearch. The commit doesn't rename the files in the `src/bin` directory as git doesn't detect `git mv`, so that change will be done in a subsequent commit.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors o.e.Version to o.opensearch.Version. This is retained in a
single commit to serve as a reference for re-versioning the opensearch codebase
from legacy 7.10 to 1.0.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the o.e.cli and o.e.client packages from elasticsearch to
o.opensearch.cli and o.opensearch.client packages in the server module,
respectively.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
Refactor the `libs/cli` module to rename the package name from `org.elasticsearch.cli` to `org.opensearch.cli` as part of the rename to OpenSearch work.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Plugins CLI - disable installing official plugins by name.
Currently the plugin cli allows installation of a plugin by name in which case it downloads the plugin artifacts from the official elastic artifacts repository.
We will enable it once we have the new official artifacts download URL (Tracking Issue: #100)
Signed-off-by: Peter Nied <petern@amazon.com>
* Replace compile configuration usage with api (#58451)
- Use java-library instead of plugin to allow api configuration usage
- Remove explicit references to runtime configurations in dependency declarations
- Make test runtime classpath input for testing convention
- required as java library will by default not have build jar file
- jar file is now explicit input of the task and gradle will ensure its properly build
* Fix compile usages in 7.x branch
The plugin installer currently checks the fake plugins installed contain
a single jar file. This commit adds another test for a plugin which
contains multiple jar files, ensuring all jars exist in the installed
plugin.
* Remove usage of deprecated testCompile configuration
* Replace testCompile usage by testImplementation
* Make testImplementation non transitive by default (as we did for testCompile)
* Update CONTRIBUTING about using testImplementation for test dependencies
* Fail on testCompile configuration usage
Guava was removed from Elasticsearch many years ago, but remnants of it
remain due to transitive dependencies. When a dependency pulls guava
into the compile classpath, devs can inadvertently begin using methods
from guava without realizing it. This commit moves guava to a runtime
dependency in the modules that it is needed.
Note that one special case is the html sanitizer in watcher. The third
party dep uses guava in the PolicyFactory class signature. However, only
calling a method on the PolicyFactory actually causes the class to be
loaded, a reference alone does not trigger compilation to look at the
class implementation. There we utilize a MethodHandle for invoking the
relevant method at runtime, where guava will continue to exist.
Change how we format exceptions to only wrap them as necessary. While
the config's overall philosophy is to put items one-per-line when
wrapping, in practice this is a little cumbersome for exception lists.
When installing plugins from remote sources, either the Elastic download
service, or maven, a checksum file is downloaded and checked against the
downloaded zip. The current format for official plugins is to use a
sha512 checksum which includes the zip filename. This format matches
that from sha512sum, and allows using the --check argument there to
verify the checksum manually. However, when generating checksum files
with maven and gradle, the filename is not included.
This commit relaxes the requirement the filename existing within the
sha512 checksum file for maven plugins. We continue to strictly enforce
official plugins have the existing format of the file.
closes#52413
Now that the FIPS 140 security provider is simply a test dependency
we don't need the thirdPartyAudit exceptions, but plugin-cli and
transport-netty4 do need jarHell disabled as they use the non fips
BouncyCastle security provider as a test dependency too.
When installing multiple plugins at once, this commit changes the
behavior to report installed plugins as we go. In the case of failure,
we emit a message that we are rolling back any plugins that were
installed successfully, and also that they were successfully rolled
back. In the case a plugin is not successfully rolled back, we report
this clearly too, alerting the user that there might still be state on
disk they would have to clean up.
This commit allows the plugin installer to install multiple plugins in a
single invocation. The installation will be treated as a transaction, so
that all of the plugins are install successfully, or none of the plugins
are installed.
This fixes a regression introduced in #42042. The logic here was
mistakenly inverted such that we only run these tests in a FIPS JVM
which is the opposite of what we intend.
Backport of #48849. Update `.editorconfig` to make the Java settings the
default for all files, and then apply a 2-space indent to all `*.gradle`
files. Then reformat all the files.
This commit introduces a consistent, and type-safe manner for handling
global build parameters through out our build logic. Primarily this
replaces the existing usages of extra properties with static accessors.
It also introduces and explicit API for initialization and mutation of
any such parameters, as well as better error handling for uninitialized
or eager access of parameter values.
Closes#42042
This is the Java side of https://github.com/elastic/ml-cpp/pull/593
with a fallback so that ml-cpp bundles with either the
new or old directory structure work for the time being.
A few days after merging the C++ changes a followup to
this change will be made that removes the fallback.
Most of our CLI tools use the Terminal class, which previously did not provide methods for writing to standard output. When all output goes to standard out, there are two basic problems. First, errors and warnings are "swallowed" in pipelines, making it hard for a user to know when something's gone wrong. Second, errors and warnings are intermingled with legitimate output, making it difficult to pass the results of interactive scripts to other tools.
This commit adds a second set of print commands to Terminal for printing to standard error, with errorPrint corresponding to print and errorPrintln corresponding to println. This leaves it to developers to decide which output should go where. It also adjusts existing commands to send errors and warnings to stderr.
Usage is printed to standard output when it's correctly requested (e.g., bin/elasticsearch-keystore --help) but goes to standard error when a command is invoked incorrectly (e.g. bin/elasticsearch-keystore list-with-a-typo | sort).
This commit applies a normalization process to environment paths, both
in how they are stored internally, also their settings values. This
normalization is done via two means:
- we make the paths absolute
- we remove redundant name elements from the path (what Java calls
"normalization")
This change ensures that when we compare and refer to these paths within
the system, we are using a common ground. For example, prior to the
change if the data path was relative, we would not compare it correctly
to paths from disk usage. This is because the paths in disk usage were
being made absolute.
Today when checksumming a plugin zip during plugin install, we read all
of the bytes of the zip into memory at once. When trying to run the
plugin installer on a small heap (say, 64 MiB), this can lead to the
plugin installer running out of memory when checksumming large
plugins. This commit addresses this by reading the plugin bytes in 8 KiB
chunks, thus using a constant amount of memory independent of the size
of the plugin.
This change makes the process of verifying the signature of
official plugins FIPS 140 compliant by defaulting to use the
BouncyCastle FIPS provider and adding a dependency to bcpg-fips
that implement parts of openPGP in a FIPS compliant manner.
In already FIPS 140 enabled environments that use the
BouncyCastle FIPS provider, the bcfips dependency is redundant
but doesn't cause an issue as it will be added only in the classpath
of the cli-tools
This is a backport of #44224
Vacha
Andriy Redko
Nick Knize
Sarat Vemulapalli
Nick Knize
Rabi Panda
Nick Knize
Ioannis Kakavas
Rene Groeschke
Ryan Ernst
Rory Hunter
Jason Tedor
Mark Vieira
David Roberts
William Brafford