Commit Graph

1049 Commits

Author SHA1 Message Date
Andriy Redko d8a1ba6912
[CVE-2020-36518] Update jackson-databind to 2.13.2.2 (#2599)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-03-29 12:24:37 -04:00
Peter Nied d43235c5cf
Move Jackson-databind to 2.13.2 (#2548)
Resolves CVE-2020-36518

Signed-off-by: Peter Nied <petern@amazon.com>
2022-03-21 18:44:13 -05:00
Tianli Feng 641350b0b6
Replace 'master' with 'cluster_manager' in 'GET Cat Nodes' API (#2441)
* Replace 'master' with 'cluster_manager' in the table header of [GET Cat Nodes](https://opensearch.org/docs/latest/opensearch/rest-api/cat/cat-nodes/) API, to promote inclusive language.
* Add `cm` as the alias for the table header `cluster_manager`
* Add `master` as the alias for the table header `cluster_manager`, for keeping compatibility when using `GET _cat/nodes?v&h=master` to show the specific column only.
* Add YAML rest test

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:04:49 -07:00
Tianli Feng 19eadb46ba
Deprecate setting 'cluster.initial_master_nodes' and introduce the alternative setting 'cluster.initial_cluster_manager_nodes' (#2463)
* Deprecate setting cluster.initial_master_nodes, and add setting cluster.initial_cluster_manager_nodes

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 14:55:49 -07:00
Nick Knize 05a5819243
[Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30 (#2487)
* [Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30

Upgrades from Lucene 9.0.0 to 9.1.0-snapshot-ea989fe8f30 in preparation for
9.1.0 GA.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Add spanishplural token filter

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix KNOWN_TOKENIZERS

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-16 15:47:25 -04:00
Nick Knize 006c832c5f
[Upgrade] Lucene 9.0.0 release (#1109)
This commit upgrades the core codebase from Lucene 8.10.1 to
lucene 9.0.0. It includes all necessary refactoring of features and
API changes when upgrading to a new major Lucene release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Andriy Redko <drreta@gmail.com>
2022-03-15 15:48:13 -05:00
Daniel Doubrovkine (dB.) 044f53630a
Set target and source compatibility to 11, required by Lucene 9. (#2407)
* Set target and source compatibility to 11, required by Lucene 9.

Signed-off-by: dblock <dblock@dblock.org>

* Uncomment commented code in #2321 for killing child processes that uses JDK9+ ProcessInfo.

Signed-off-by: dblock <dblock@dblock.org>

* Set distribution checker target JDK compatibility to 11.

Signed-off-by: dblock <dblock@dblock.org>

* Supress processing warnings.

Signed-off-by: dblock <dblock@dblock.org>
2022-03-08 14:48:51 -05:00
dependabot[bot] 4395ed560f
Bump guava in /distribution/tools/upgrade-cli (#2383)
Bumps [guava](https://github.com/google/guava) from 31.0.1-jre to 31.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-07 11:48:38 -05:00
dependabot[bot] 72c5d81187
Bump guava in /distribution/tools/keystore-cli (#2384)
Bumps [guava](https://github.com/google/guava) from 31.0.1-jre to 31.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-07 11:48:18 -05:00
dependabot[bot] 75e837d056
Bump guava from 31.0.1-jre to 31.1-jre in /distribution/tools/plugin-cli (#2387)
Bumps [guava](https://github.com/google/guava) from 31.0.1-jre to 31.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-07 11:48:00 -05:00
Owais Kazi 9224537704
Updated the url for docker distribution (#2325)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-03-04 18:58:13 -06:00
Andriy Redko ae52008463
Fixing the --release flag usage for javac (#2343) (#2352)
* Fixing the --release flag usage for javac (#2343)

* Fixing the --release flag usage for javac

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fixing the --html5 flag usage for javadoc

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix java-version-checker source/target compatibility settings (#2354)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-03-04 17:44:52 -05:00
dependabot[bot] 9780fc6199
Bump bc-fips from 1.0.2.1 to 1.0.2.3 in /distribution/tools/plugin-cli (#2276)
* Bump bc-fips from 1.0.2.1 to 1.0.2.3 in /distribution/tools/plugin-cli

Bumps bc-fips from 1.0.2.1 to 1.0.2.3.

---
updated-dependencies:
- dependency-name: org.bouncycastle:bc-fips
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
2022-02-28 12:16:34 -08:00
dependabot[bot] 5d0b015527
Bump jimfs from 1.1 to 1.2 in /distribution/tools/keystore-cli (#2272)
Bumps [jimfs](https://github.com/google/jimfs) from 1.1 to 1.2.
- [Release notes](https://github.com/google/jimfs/releases)
- [Commits](https://github.com/google/jimfs/compare/v1.1...v1.2)

---
updated-dependencies:
- dependency-name: com.google.jimfs:jimfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-28 10:41:33 -08:00
joshpalis b251d2b565
Install plugin command help (#2193)
* edited opensearch-plugin install help output to include plugin URL

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* fixed unit test for plugin install help output by correctly identifying the beginning og the non-option argument list

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* added comments to install plugins help non option argument ouput unit test

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* fixed format violation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* added additional details on valid plugin ids and how to use plugin URLs

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* added additional information to plugin install help output

Signed-off-by: Joshua Palis <jpalis@amazon.com>

Co-authored-by: Joshua Palis <jpalis@amazon.com>
2022-02-24 08:58:12 -08:00
dependabot[bot] d7b8b323a5
Bump guava in /distribution/tools/keystore-cli (#2118)
Bumps [guava](https://github.com/google/guava) from 30.1.1-jre to 31.0.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-21 13:22:36 -08:00
dependabot[bot] 268b3b51f0
Bump jimfs from 1.1 to 1.2 in /distribution/tools/plugin-cli (#2182)
Bumps [jimfs](https://github.com/google/jimfs) from 1.1 to 1.2.
- [Release notes](https://github.com/google/jimfs/releases)
- [Commits](https://github.com/google/jimfs/compare/v1.1...v1.2)

---
updated-dependencies:
- dependency-name: com.google.jimfs:jimfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-21 12:03:28 -08:00
dependabot[bot] 6fcd34c86d
Bump nebula.ospackage-base from 9.0.0 to 9.1.1 in /distribution/packages (#2117)
Bumps nebula.ospackage-base from 9.0.0 to 9.1.1.

---
updated-dependencies:
- dependency-name: nebula.ospackage-base
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-18 15:28:01 -08:00
dependabot[bot] b31da7ccad
Bump guava in /distribution/tools/plugin-cli (#2119)
Bumps [guava](https://github.com/google/guava) from 30.1.1-jre to 31.0.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-17 15:07:13 -08:00
dependabot[bot] 17189bec85
Bump guava in /distribution/tools/upgrade-cli (#2141)
Bumps [guava](https://github.com/google/guava) from 30.1.1-jre to 31.0.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-17 14:24:32 -08:00
Suraj Singh 892801a074
Remove type mapping from document index API (#2026)
* Remove doc type specific indexing APIs and relevant changes

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Remove type param from yml and integ test files

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-09 13:34:10 -06:00
Lukáš Vlček 270c59f523
Fix AssertionError message (#2044)
Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
2022-02-02 16:58:25 -06:00
Andriy Redko c8ac037389
[FEATURE] Add OPENSEARCH_JAVA_HOME env to override JAVA_HOME (#2001)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-02 13:55:22 -05:00
Andriy Redko 6c2f01a045
[BUG] Docker distribution builds are failing. Switching to http://vault.centos.org (#2024)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-01 12:42:48 -06:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
sharp-pixel 1bb8864178
Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions. (#1668)
* Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>

* Fix dangling HTML tags in javadoc to solve build issues with JDK 17.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>
2022-01-06 12:39:13 -05:00
Tianli Feng db23f72a2a
Update FIPS API libraries of Bouncy Castle (#1853)
* Update bc-fips to 1.0.2.1

Signed-off-by: Tianli Feng <ftl94@live.com>

* Update bcpg-fips to 1.0.5.1

Signed-off-by: Tianli Feng <ftl94@live.com>

* Update bctls-fips to 1.0.12.2

Signed-off-by: Tianli Feng <ftl94@live.com>

* Use the unified bouncycastle version for bcpkix-jdk15on in HDFS testing fixture

Signed-off-by: Tianli Feng <ftl94@live.com>
2022-01-05 16:59:55 -08:00
Andriy Redko 546eacc3dd
Gradle clean failing after a failed gradle check, folders created by Docker under 'root' user (#1726)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-18 10:57:04 -05:00
Andriy Redko 069e229b29
Support JDK 18 EA builds (#1710)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-13 12:42:25 -05:00
Andriy Redko 8bf6fa6b4f
Support Gradle 7 (#1609)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-29 14:34:06 -05:00
Vacha bcfb57c06a
Upgrade dependency (#1571)
* Upgrading guava, commons-io and apache-ant dependencies

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding failureaccess since guava needs it

Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-18 13:38:49 -05:00
Andrew Ross 546ab21989
Use a non-default port for upgrade-cli unit tests (#1512)
I observed a [test failure][1] that I believe was caused by the fact
that an OpenSearch server happened to be running at localhost:9200 when
these unit tests were executed. The code under test has logic to try to
connect to localhost:9200 and then fall back to defaults if that port is
not open. The tests expect these defaults and will fail if different
data is returned. The change here is to use a non-default port to make
it very unlikely that a real instance will be running at the non-default
port. I don't know why an OpenSearch service happened to be running
during the linked test failure, but I think making these unit tests more
independent and isolated is helpful no matter the underlying cause in
this case.

[1]: https://fork-jenkins.searchservices.aws.dev/job/OpenSearch_CI/job/PR_Checks/job/Gradle_Check/975/artifact/gradle_check_975.log/*view*/

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-11-05 09:52:16 -04:00
Andriy Redko f469d53cff
[BUG] SymbolicLinkPreservingUntarTransform fails on Windows (#1433)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-26 09:45:26 -05:00
Romain Tartière ea0fe7bfae
Allow building on FreeBSD (#1091)
* Allow building on FreeBSD

With this set of change, we are able to successfuly run:

```
./gradlew publishToMavenLocal -Dbuild.snapshot=false
```

This step is used in the OpenSearch repository context when building
plugins in the current state of the CI.

While here, reorder OS conditions alphabetically.

Before building, the openjdk14 package was installed and the environment
was adjusted to use it:

```
sudo pkg install openjdk14
export JAVA_HOME=/usr/local/openjdk14/
export PATH=$JAVA_HOME/bin:$PATH
```

Signed-off-by: Romain Tartière <romain@blogreen.org>

* Unbreak CI with FreeBSD support

Signed-off-by: dblock <dblock@dblock.org>

Co-authored-by: dblock <dblock@dblock.org>
2021-10-14 14:42:28 -04:00
Andriy Redko 3779576c51
Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests (#1358)
* Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Using -Djava.security.egd=file:/dev/urandom explicitly for cli tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-13 17:25:48 -04:00
Andriy Redko e9635d6bfe
Replace securemock with mock-maker (test support), update Mockito to 3.12.4 (#1332)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-10 14:18:54 -04:00
Xue Zhou 0ab8e34022
Rename artifact produced by the build to include -min (#1251)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-09-21 19:43:35 -04:00
Andriy Redko cdbc84f09d
Update Jackson to 2.12.5 (#1247)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-21 18:33:20 -04:00
Nick Knize aecc7bd005
[Bug] Fix InstallPluginCommand to use proper key signatures (#1233)
The public key has changed since the initial release. This commit fixes the
public key and uses the .sig files that are published to the artifacts site.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-09-10 14:58:31 -07:00
Sven R 268d93b375
FreeBSD Java support (#1014)
Signed-off-by: hackacad <admin@hackacad.net>
2021-09-07 18:17:58 -04:00
Minal Shah a43e6cf424
Fixed copyright to OpenSearch (#1175) 2021-08-31 12:55:28 -04:00
Nick Knize ff7e7904ca
[DEPRECATE] SimpleFS in favor of NIOFS (#1073)
Lucene 9 removes support for SimpleFS File System format. This commit deprecates
the SimpleFS format in favor of NIOFS.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-19 17:56:55 -05:00
Sven R dcd9cef56c
alt bash path support (#1047)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-06 11:09:29 -04:00
Sven R 0fd14e8c3c
alt bash path support (#1013)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-03 09:10:30 -04:00
Vacha 25f0b1e6bd
Validation for official plugins for upgrade tool (#973)
Add validation to check for official plugins during the plugins installation task for the upgrade tool.

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-27 09:01:10 -07:00
Vacha 19e54d6624
Allowing custom folder name for plugin installation (#848)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-22 04:28:49 +05:30
Poojita Raj 6bc4ce017a
Replace Elasticsearch docs links in scripts (#994)
Replace the docs links In scripts bin/opensearch-env and config/jvm.options, with OpenSearch docs links.

Signed-off-by: Poojita-Raj <poojiraj@amazon.com>
2021-07-21 14:23:22 -07:00
xuezhou25 e3d86baa5a
Fix opensearch-env always sources the environment from hardcoded file (#875)
distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-07-21 07:51:26 -07:00
Himanshu Setia 13a02becba
Restricting logs permissions (#966)
Currently, the permissions for opensearch logs are -rw-r-r-, which gives read access to anyone. This weak permission
structure can lead to leakage of any sensitive information (if published) in the logs. This commit restricts read
access with -rw-r--- permission.

Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-07-15 10:06:45 -07:00
Rabi Panda 4906a3c1b0
A CLI tool to assist during an upgrade to OpenSearch. (#846)
This change adds the initial version of a new CLI tool `opensearch-upgrade` as part of the OpenSearch distribution. This tool is meant for assisting during an upgrade from an existing Elasticsearch v7.10.2/v6.8.0 node to OpenSearch. It automates the process of importing existing configurations and installing of core plugins.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-07-13 09:51:02 -07:00