Commit Graph

641 Commits

Author SHA1 Message Date
Wenjun Ruan 0b1f4a2069
Rename `file` to `dir` in Environment (#2730)
* Rename `file` to `dir` in Environment

Signed-off-by: ruanwenjun <wenjun@apache.org>

* Fix compile error

Signed-off-by: ruanwenjun <wenjun@apache.org>

* fix compile error

Signed-off-by: ruanwenjun <wenjun@apache.org>
2022-04-05 09:23:08 -04:00
Sruti Parthiban 8b997c1d84
Add resource stats to task framework (#2089)
* Add resource stats to task framework

Signed-off-by: sruti1312 <srutiparthiban@gmail.com>

* Update thread resource info and add tests

Signed-off-by: sruti1312 <srutiparthiban@gmail.com>
2022-03-28 13:00:59 -04:00
Tianli Feng a85325363f
Use Hamcrest matchers and assertThat() in ReindexRenamedSettingTests (#2503)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-19 01:22:04 -07:00
Nick Knize 05a5819243
[Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30 (#2487)
* [Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30

Upgrades from Lucene 9.0.0 to 9.1.0-snapshot-ea989fe8f30 in preparation for
9.1.0 GA.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Add spanishplural token filter

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix KNOWN_TOKENIZERS

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-16 15:47:25 -04:00
Nick Knize 95d4750249
[Remove] types from translog (#2439)
Removes persisting the type in the translog since types are no longer supported.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-12 14:03:00 -06:00
Tianli Feng 63c75d1b1d
Deprecate setting 'reindex.remote.whitelist' and introduce the alternative setting 'reindex.remote.allowlist' (#2221)
* Add setting reindex.remote.allowlist, and deprecate setting reindex.remote.whitelist

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add unit test for renaming the setting reindex.remote.allowlist

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Remove system.out.println()

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Adjust format by spotlessApply task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace REMOTE_CLUSTER_WHITELIST with REMOTE_CLUSTER_ALLOWLIST

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add a unit test to test final setting value when both settings have got a value

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Rename the unit test class name

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Remove the Access modifiers public from the constant REMOTE_CLUSTER_WHITELIST

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Initialize ReindexPlugin without using the @Before method

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Rename 'unwhitelisted' to 'unallowlisted' in a yml file used for REST api testing.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-08 11:35:36 -05:00
aponb be64af2f90
Replace exclusionary words whitelist and blacklist in the places that won't impact backwards compatibility (#2178)
* Replace the exclusionary word whitelist with allowlist, and blacklist with denylist, in code commet and internal variable/method/class/package name.

Signed-off-by: Andreas <apre@gmx.at>
2022-03-04 16:34:09 -08:00
Nick Knize 0cc2c9be6a
[Remove] types from PutMappingRequest (#2335)
Remove type support from putMappingRequest, dependencies, and all tests.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-04 13:30:43 -06:00
Suraj Singh 3d5aff4b91
Remove type end-points from search and related APIs (#2263)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-03-02 13:21:00 -08:00
Nick Knize 0bd7850bed
[Remove] remaining type usage in Client and AbstractClient (#2258)
Removes type parameter from remaining prepareIndex in Client and AbstractClient.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-25 13:35:48 -06:00
Nick Knize 8b48207540
[Remove] Type from Client.prepare(Index,Delete,Update) (#2253)
Removes the type parameter from Client.prepare(Index,Delete,Update) and
everywhere it's used throughout the codebase except for prepareIndex(index,
type, id) which is removed in a follow up.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-24 21:20:03 -06:00
Nick Knize d892c51d66
[Remove] Types from DocWrite Request and Response (#2239)
Removes type support from DocWrite Request and Response, all derived classes,
and all places used.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-24 15:57:44 -06:00
Nick Knize 3445befd36
[Remove] types from SearchHit and Explain API (#2205)
* [Remove] types from SearchHit and Explain API

Removes type support from SearchHit and Explain API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix failing tests

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* move TYPES_DEPRECATION_MESSAGE from RestExplainAction to FullClusterRestartIT

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix FullClusterRestartIT

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix failing tests

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix testSoftDelete

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-23 11:40:30 -08:00
Nick Knize 1b571ece28
[Remove] type support from Bulk API (#2215)
Removes all support for type parameter from the Bulk API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-23 11:53:27 -06:00
Nick Knize 3e9a420533
[Remove] 6x skip from yml (#2153)
Remove all 6x skip versions from yaml tests since mix 2.0 and 6x clusters are
not supported.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-22 23:20:18 -06:00
Nick Knize 2e3f410957
[Remove] Types from GET/MGET (#2168)
Removes type support from the get and mget transport and rest actions along
with removal from ShardGetService.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-21 11:37:27 -06:00
Nick Knize 7fe642fda5
[Remove] Type from Search Internals (#2109)
With types deprecation the type support is removed from internal search API
(SearchRequest and QueryShardContext).

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-17 16:17:31 -06:00
Andriy Redko 385b268bc0
Update Mockito to 4.2.x (#1830)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-03 12:00:45 -05:00
Rabi Panda 06cbc47136
Remove LegacyESVersion.V_6_7_x constants (#1807)
This commit removes all usages of the `LegacyESVersion.V_6_7_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-02 14:09:55 -08:00
Rabi Panda 24d0c9b818
Remove transport client from tests. (#1809)
Transport client has been deprecated. In this commit, we are removing all its usages from the tests.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-27 17:43:18 -06:00
Megha Sai Kavikondala 22bfadf324
Adding a cancelled field to tell if a cancellable task is cancelled (#1732)
* Changes made by adding the Task indicator and the IllegalArgs.

Signed-off-by: meghasaik <kavmegha@amazon.com>

* Changes made by changing the String message for uncancellable task.

Signed-off-by: meghasaik <kavmegha@amazon.com>
2021-12-16 09:49:49 -05:00
Vacha e66ea2c4f3
Avoid logging duplicate deprecation warnings multiple times (#1660)
* Avoid logging duplicate deprecation warnings multiple times

Signed-off-by: Vacha <vachshah@amazon.com>

* Fixes test failures

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding deprecation logger tests

Signed-off-by: Vacha <vachshah@amazon.com>

* Using ConcurrentHashMap keySet

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-12-15 15:26:44 -08:00
Nick Knize b74d71fb74
Remove 6.0.* version constants (#1658)
This PR removes LegacyESVersion.V_6_0_* constants including all pre-release
versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-08 13:37:21 -06:00
Andriy Redko 8bf6fa6b4f
Support Gradle 7 (#1609)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-29 14:34:06 -05:00
Nick Knize 53334b2ce4
Upgrade to Lucene 8.10.1 (#1440)
This commit upgrades to the latest release of lucene 8.10

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-10-28 10:06:53 -05:00
Owais Kazi 37ac3788a3
Run spotless and exclude checkstyle on modules module (#1442)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-27 13:24:43 -07:00
Xue Zhou 574e42c31b
Remove old ES libraries used in reindex due to CVEs (#1359)
This commit removes old ES libraries version 090 and 176 due to CVE

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-10-20 11:51:45 -05:00
Andriy Redko b6c8bdf872
Drop mocksocket in favour of custom security manager checks (tests only) (#1205)
* Drop mocksocket in favour of custom security manager checks (tests only)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Slightly relaxed host checks to allow all local addresses

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-16 17:21:47 -04:00
Andriy Redko 0b773c2cec
Fix org.opensearch.index.reindex.ReindexRestClientSslTests#testClientSucceedsWithCertificateAuthorities - javax.net.ssl.SSLPeerUnverifiedException (#1212)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-08 09:08:05 -04:00
Sooraj Sinha a428e6f043
Pass interceptor to super constructor (#876)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-07-05 11:13:36 +05:30
Nick Knize d821b15a6b
[BWC] fix mixedCluster and rolling upgrades (#775)
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: Shweta Thareja <tharejas@amazon.com>
2021-05-28 00:42:57 -05:00
Sooraj Sinha a8b8a96036
Remove URL content from Reindex error response (#630)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-05-07 07:33:16 -07:00
Sooraj Sinha 181ee8a211
Add Remote Reindex SPI extension (#547)
This change extends the remote reindex SPI to allow adding a custom interceptor.
This interceptor can be plugged in to perform any processing on the request or response.

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-05-06 09:38:25 -07:00
Nick Knize 6e04778d0c
[Versioning] Fix Version.fromString logic for legacy version (#604)
This commit fixes the Version.fromString logic to identify legacy versions. It
also adds an optional "distribution" field to the MainRespose for OpenSearch
version 1.0.0+. Any preceeding versions that do not contain the distribution
label will be handeled as legacy versions appropriately.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-22 23:04:44 -05:00
Robert Muir 5474e8d094
Speed ups to test suite and precommit tasks. (#580)
Use C1 compiler only for short-lived tasks and unit test execution. Tone
down some of the slowest unit tests.

Signed-off-by: Robert Muir <rmuir@apache.org>
2021-04-20 09:02:45 -05:00
Nick Knize 0ba0e7cc26
[Versioning] Rebase to OpenSearch version 1.0.0 (#555)
This commit rebases the versioning to OpenSearch 1.0.0

Co-authored-by: Rabi Panda <adnapibar@gmail.com>

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-15 17:06:47 -05:00
Nick Knize ee6d15e26a
[License] Add SPDX License Header to security policies (#531)
This commit adds the SPDX license header and modifications copyright to security
policy files.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-12 22:59:36 -05:00
Nick Knize 9168f1fb43
[License] Add SPDX and OpenSearch Modification license header (#509)
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-09 14:28:18 -05:00
Rabi Panda 0bdd1293c1
Use alternate example data in OpenSearch test cases. (#454)
This commit updates some of the sample test data used in test cases in OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-25 08:52:07 -07:00
Rabi Panda 2e3055c9e2
Fix more failing tests as a result of renaming (#457)
This commit fixes some more renaming issues and as a result fixes the failing tests,

* :qa:logging-config:test 
* :example-plugins:painless-whitelist:yamlRestTest
* :modules:reindex:test

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-24 09:33:05 -07:00
Nick Knize 1babf43777
[Rename] Fix env variables and old es maven repo (#439)
This commit fixes some straggling es. environment variables along with a
reference to old elasticsearch versions from an opensearch maven repo that
doesn't exist.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-22 10:17:54 -07:00
Nick Knize 7051167c83 [Rename] remaining elasticsearch pass 1 (#416)
This commit refactors instances of 'elasticsearch' with opensearch everywhere
except references to issues, and other places needed to test compatibility with
old elasticsearch clusters.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda eddfe6760d [Rename] Fix issues for gradle precommit task. (#418)
Fix miscellaneous issues identified during `gradle precommit`. These issues are the side effects of the renaming to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda d1e070c92b [Rename] Fix import issues in tests. (#414)
This commit fixes the import issues in already refactored test packages.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 127243b83b [Rename] Environment variables in entire repository. (#405)
Rename all environment variables with prefix 'ES_' to 'OPENSEARCH_'.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda df11cc9de4 [Rename] Fix gradle build as part of the renaming process. (#397)
This commit fixes the currently broken gradle build resulted from the renaming work. It reverts a few dependencies and comments out the `opensearch_distibutions` task which is currently failing for some builds. We will address these separately in the future once we have a working build.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda da8a35f464 [Rename] Fix imports in the modules directory. (#386)
Fix import issues in the modules directory resulting from the refactoring.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Himanshu Setia 8e711ab922 [Rename] Refactor o.e.watcher (#399)
This commit refactors the o.e.watcher package.  References throughout the codebase are also refactored.

Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 13f6d23e40 [Rename] Property and metadata keys with prefix es. (#389)
Rename all property and metadata keys with prefix 'es.' to 'opensearch.'.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize f75803e1aa [Rename] Second pass clean up of imports and variable names in server module (#382)
This commit cleans up imports, variable names, comments, and other misc usages
of ES with the new OpenSearch name.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00