Commit Graph

1794 Commits

Author SHA1 Message Date
Yoann Rodière b5d5616d44
Update commons-logging to 1.2 (#2806)
* Upgrade to Apache Commons Logging 1.2

Signed-off-by: Yoann Rodière <yoann@hibernate.org>

* Clarify that Apache HTTP/commons-* dependencies are not just for tests

Signed-off-by: Yoann Rodière <yoann@hibernate.org>
2022-04-08 16:43:51 -04:00
Nick Knize cb238aae61
Bump version to 3.0.0 (#2556)
Bump version to 3.0.0

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-04 17:40:31 -05:00
Nick Knize 86eb24d27f
[Remove] types from rest-api-spec endpoints (#2689)
Removes types from rest-api-spec endpoints

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-31 23:08:19 -05:00
Sruti Parthiban 8b997c1d84
Add resource stats to task framework (#2089)
* Add resource stats to task framework

Signed-off-by: sruti1312 <srutiparthiban@gmail.com>

* Update thread resource info and add tests

Signed-off-by: sruti1312 <srutiparthiban@gmail.com>
2022-03-28 13:00:59 -04:00
Peter Nied d43235c5cf
Move Jackson-databind to 2.13.2 (#2548)
Resolves CVE-2020-36518

Signed-off-by: Peter Nied <petern@amazon.com>
2022-03-21 18:44:13 -05:00
Tianli Feng ea31483126
Add a new node role 'cluster_manager' as the alternative for 'master' role and deprecate 'master' role (#2424)
Add a new node role `cluster_manager`, as the replacement for `master` role, that used in setting `node.roles: [ master ]`. 
They have got the same functionality, but can NOT be assigned to `node.roles` together.
* Add `CLUSTER_MANAGER_ROLE` in `DiscoveryNodeRole` class, and deprecate `MASTER_ROLE`
* Remove `MASTER_ROLE` from the `DiscoveryNodeRole.BUILT_IN_ROLES`, and temporarily load the role by the existing method `DiscoveryNode.setAdditionalRoles`
* Add a method `validateRole()` in `DiscoveryNodeRole` class, it's used to validate a specific role is compatible with the other roles to be assigned to a node together.
* Add deprecation message when assigning `master` role in setting `node.roles`
* Replace most `MASTER_ROLE` with `CLUSTER_MANAGER_ROLE` in current unit and integration tests
* Add new unit and integration tests to validate `CLUSTER_MANAGER_ROLE` and `MASTER_ROLE` can be treated as the same.

More explanation:
* New node will have "cluster_manager", "data", and "ingest" roles by default. Which means the default value of setting "node.roles" will be `["cluster_manager","data","ingest"]`, instead of `["master","data","ingest"]`
* "cluster_manager" role will be treated as "master" role in the OpenSearch node of previous versions.
* "cluster_manager” role and "master" role have got the same abbreviation name: "m"

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:06:06 -07:00
Nick Knize 05a5819243
[Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30 (#2487)
* [Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30

Upgrades from Lucene 9.0.0 to 9.1.0-snapshot-ea989fe8f30 in preparation for
9.1.0 GA.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Add spanishplural token filter

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix KNOWN_TOKENIZERS

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-16 15:47:25 -04:00
Nick Knize b00b3ce2b5
[Remove] Multiple Types from IndexTemplateMetadata (#2400)
Removes multi-type support from IndexTemplateMetadata so that instead of holding
a map of multiple types to mappings, it only returns a single mapping for a
single type. Also removes type from documentMapper() method to avoid any
accidental NullPointerExceptions in the internal mapping retrieval.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-10 22:36:08 -06:00
Daniel Doubrovkine (dB.) 044f53630a
Set target and source compatibility to 11, required by Lucene 9. (#2407)
* Set target and source compatibility to 11, required by Lucene 9.

Signed-off-by: dblock <dblock@dblock.org>

* Uncomment commented code in #2321 for killing child processes that uses JDK9+ ProcessInfo.

Signed-off-by: dblock <dblock@dblock.org>

* Set distribution checker target JDK compatibility to 11.

Signed-off-by: dblock <dblock@dblock.org>

* Supress processing warnings.

Signed-off-by: dblock <dblock@dblock.org>
2022-03-08 14:48:51 -05:00
Nick Knize c3712a51b3
[Remove] include_type_name from HLRC (#2397)
Removes include_type_name from the high level reset client along with relevant
deprecated methods in IndicesClient. All tests are updated to remove the
parameter from the rest requests along with various toXContent methods that are
no longer required.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-08 11:30:27 -06:00
Tianli Feng 63c75d1b1d
Deprecate setting 'reindex.remote.whitelist' and introduce the alternative setting 'reindex.remote.allowlist' (#2221)
* Add setting reindex.remote.allowlist, and deprecate setting reindex.remote.whitelist

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add unit test for renaming the setting reindex.remote.allowlist

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Remove system.out.println()

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Adjust format by spotlessApply task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace REMOTE_CLUSTER_WHITELIST with REMOTE_CLUSTER_ALLOWLIST

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add a unit test to test final setting value when both settings have got a value

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Rename the unit test class name

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Remove the Access modifiers public from the constant REMOTE_CLUSTER_WHITELIST

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Initialize ReindexPlugin without using the @Before method

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Rename 'unwhitelisted' to 'unallowlisted' in a yml file used for REST api testing.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-08 11:35:36 -05:00
aponb be64af2f90
Replace exclusionary words whitelist and blacklist in the places that won't impact backwards compatibility (#2178)
* Replace the exclusionary word whitelist with allowlist, and blacklist with denylist, in code commet and internal variable/method/class/package name.

Signed-off-by: Andreas <apre@gmx.at>
2022-03-04 16:34:09 -08:00
Nick Knize 0cc2c9be6a
[Remove] types from PutMappingRequest (#2335)
Remove type support from putMappingRequest, dependencies, and all tests.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-04 13:30:43 -06:00
Nick Knize 897f4e7295
[Remove] deprecated getMapping API from IndicesClient (#2262)
Removes the deprecated types based get, getMapping, getAsync, and
getMappingAsync methods from IndicesClient. It also removes extra nesting of
mappings belong the deprecated type named object and removes the types based
methods from the affected request classes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-02 13:44:04 -06:00
Suraj Singh 223d0b2a1c
Remove type end-points from no-op bulk and search action (#2261)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-25 13:00:25 -08:00
Suraj Singh 5b0da85df8
Remove type from validate query API (#2255)
* Remove type mapping from RestValidateAction

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Spotless check apply

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Include suggested review comment

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-25 10:18:41 -08:00
Nick Knize d892c51d66
[Remove] Types from DocWrite Request and Response (#2239)
Removes type support from DocWrite Request and Response, all derived classes,
and all places used.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-24 15:57:44 -06:00
Suraj Singh cbfcad9054
Removes type mappings from mapping APIs (#2238)
Removes type mappings from GetMapping, GetFieldMapping and PutMapping APIs

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-24 11:23:36 -08:00
Nick Knize 3445befd36
[Remove] types from SearchHit and Explain API (#2205)
* [Remove] types from SearchHit and Explain API

Removes type support from SearchHit and Explain API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix failing tests

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* move TYPES_DEPRECATION_MESSAGE from RestExplainAction to FullClusterRestartIT

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix FullClusterRestartIT

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix failing tests

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix testSoftDelete

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-23 11:40:30 -08:00
Nick Knize 1b571ece28
[Remove] type support from Bulk API (#2215)
Removes all support for type parameter from the Bulk API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-23 11:53:27 -06:00
dependabot[bot] 53ccd7c341
Bump commons-math3 from 3.2 to 3.6.1 in /client/benchmark (#2139)
Bumps commons-math3 from 3.2 to 3.6.1.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-math3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-21 15:29:57 -08:00
Nick Knize c9f7c37367
[Remove] Type Specific Index Stats (#2198)
Removes type specific index stats since only one type is allowed.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-21 16:06:43 -06:00
Nick Knize 2e3f410957
[Remove] Types from GET/MGET (#2168)
Removes type support from the get and mget transport and rest actions along
with removal from ShardGetService.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-21 11:37:27 -06:00
Nick Knize 7fe642fda5
[Remove] Type from Search Internals (#2109)
With types deprecation the type support is removed from internal search API
(SearchRequest and QueryShardContext).

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-17 16:17:31 -06:00
dependabot[bot] 9689a27b63
Bump com.diffplug.spotless from 5.6.1 to 6.2.0 (#1919)
* Bump com.diffplug.spotless from 5.6.1 to 6.2.0

Bumps com.diffplug.spotless from 5.6.1 to 6.2.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-16 22:08:40 -08:00
Nick Knize c0c7d7aedb
[Remove] Type parameter from TermVectors API (#2104)
Remove document type support from the TermVectors API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-15 14:49:08 -06:00
Suraj Singh 892801a074
Remove type mapping from document index API (#2026)
* Remove doc type specific indexing APIs and relevant changes

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Remove type param from yml and integ test files

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-09 13:34:10 -06:00
Martin Gaievski 309397520c
Adding support for JDK17 and removing JDK8 (#2025)
* Adding support for JDK17 and removing JDK8

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Merge overlaping PR, bumping min java version to 11

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Removing references to JDK8 from dev guide

Signed-off-by: Martin Gaievski <gaievski@amazon.com>
2022-02-02 20:59:10 -05:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
Nick Knize 0791c88af1
[Remove] Deprecated Synced Flush API (#1761)
Remove the deprecated sync flush API which was replaced by sequence
number and retention lease mechanisms and no longer used in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 22:18:48 -06:00
Andriy Redko 385b268bc0
Update Mockito to 4.2.x (#1830)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-03 12:00:45 -05:00
Rabi Panda 7c73ed8d4c
Remove deprecated transport client (#1781)
This commit removes all usage of transport client in OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-28 11:28:51 -06:00
Megha Sai Kavikondala 22bfadf324
Adding a cancelled field to tell if a cancellable task is cancelled (#1732)
* Changes made by adding the Task indicator and the IllegalArgs.

Signed-off-by: meghasaik <kavmegha@amazon.com>

* Changes made by changing the String message for uncancellable task.

Signed-off-by: meghasaik <kavmegha@amazon.com>
2021-12-16 09:49:49 -05:00
Vacha e66ea2c4f3
Avoid logging duplicate deprecation warnings multiple times (#1660)
* Avoid logging duplicate deprecation warnings multiple times

Signed-off-by: Vacha <vachshah@amazon.com>

* Fixes test failures

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding deprecation logger tests

Signed-off-by: Vacha <vachshah@amazon.com>

* Using ConcurrentHashMap keySet

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-12-15 15:26:44 -08:00
Nick Knize b74d71fb74
Remove 6.0.* version constants (#1658)
This PR removes LegacyESVersion.V_6_0_* constants including all pre-release
versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-08 13:37:21 -06:00
Ryan Bogan 7496f8058b
Added more detailed logging for SSLHandshakeException (#1602)
* Added more detailed logging for SSLHandshakeException

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Fixed spotless errors for logging change

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Changed message to include original error and limited to specific instance

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Applied troubleshooting message to all instances of SSlHandshakeException

Signed-off-by: Ryan Bogan <rbogan@amazon.com>
2021-12-06 08:05:28 -05:00
Jan Baudisch 4eaa60ce60
Enable RestHighLevel-Client to set parameter require_alias for bulk index and reindex requests (#1533)
Signed-off-by: Jan Baudisch <jan.baudisch.libri@gmail.com>

Co-authored-by: Jan Baudisch <jan.baudisch.libri@gmail.com>
2021-11-22 12:50:35 -05:00
Owais Kazi 33e70a9886
Run spotless and exclude checkstyle on client module (#1392)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-21 13:13:38 -07:00
Andriy Redko e9635d6bfe
Replace securemock with mock-maker (test support), update Mockito to 3.12.4 (#1332)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-10 14:18:54 -04:00
Andriy Redko cdbc84f09d
Update Jackson to 2.12.5 (#1247)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-21 18:33:20 -04:00
Andriy Redko f60d093c63
Drop mocksocket & securemock dependencies from sniffer and rest client (no needed) (#1174)
* Drop mocksocket & securemock dependencies from sniffer and rest client (not needed)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Removing .gitignore

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-08-31 14:18:37 -04:00
Sven R dcd9cef56c
alt bash path support (#1047)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-06 11:09:29 -04:00
Yevhen Tienkaiev f46f6950ec
Rank feature - unknown field linear (#983)
Signed-off-by: Yevhen Tienkaiev <hronom@gmail.com>
2021-07-29 14:28:29 -04:00
Vacha e17ce53eb7
Adding broken links checker (#877)
* Adding broken links checker

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Adding exclusions for links

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Correcting broken link

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Removing the benchmarks link

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-12 14:07:56 -07:00
Marc Handalian c1250c963d
Add cluster setting to spoof version number returned from MainResponse (#847)
This change adds a new cluster setting "compatibility.override_main_response_version"
that when enabled spoofs the version.number returned from MainResponse
for REST clients expecting legacy version 7.10.2.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-06-22 10:42:23 -07:00
Gregor Zurowski 0f9060761c
Fix Javadoc errors in `client/sniffer` (#802)
* Adds a gradle plugin to validate missing javadocs

Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fix Javadoc errors in client/sniffer module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Add package info to client/sniffer module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

Co-authored-by: Himanshu Setia <setiah@amazon.com>
2021-06-03 17:08:16 -07:00
Nick Knize d821b15a6b
[BWC] fix mixedCluster and rolling upgrades (#775)
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: Shweta Thareja <tharejas@amazon.com>
2021-05-28 00:42:57 -05:00
Himanshu Setia 6f893ed1cd
Merging javadoc feature branch changes to main (#715)
* Adds a gradle plugin to validate missing javadocs

Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fix Javadoc errors in module `client/rest` (#685)

* Fix Javadoc errors in client/rest module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Add package info file in client/rest module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Fix typos

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Add exception documentation to Javadoc

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Fixes precommit task configuration failures due to newly added missin… (#707)

* Fixes precommit task configuration failures due to newly added missingJavadoc task

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fixes javadoc task errors due to PR#685

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Updated CONTRIBUTING.md for info on javadocs

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Correcting licenses and naming

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Correcting version info

Signed-off-by: Himanshu Setia <setiah@amazon.com>

Co-authored-by: Gregor Zurowski <gregor@zurowski.org>
2021-05-18 13:21:41 -07:00
Rabi Panda 50abf6d066
[CVE] Upgrade dependencies to mitigate CVEs (#657)
This PR upgrade the following dependencies to fix CVEs.

- commons-codec:1.12 (->1.13) apache/commons-codec@48b6157
- ant:1.10.8 (->1.10.9) https://ant.apache.org/security.html
- jackson-databind:2.10.4 (->2.11.0) FasterXML/jackson-databind#2589
- jackson-dataformat-cbor:2.10.4 (->2.11.0) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-28491
- apache-httpclient:4.5.10 (->4.5.13) https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2020-13956
- checkstyle:8.20 (->8.29) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10782
- junit:4.12 (->4.13.1) https://github.com/junit-team/junit4/security/advisories/GHSA-269g-pwp5-87pp
- netty:4.1.49.Final (->4.1.59) https://github.com/netty/netty/security/advisories/GHSA-5mcr-gq6c-3hq2

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-05-18 11:37:24 -07:00
Daniel Doubrovkine (dB.) 6d1e6a0720
Converted all .asciidoc to .md. (#658)
Signed-off-by: dblock <dblock@amazon.com>
2021-05-05 15:04:14 -04:00