Commit Graph

2795 Commits

Author SHA1 Message Date
Daniel Doubrovkine (dB.) bd9ca7cce5
Version checks are incorrectly returning versions < 1.0.0. (#797)
* Version checks are incorrectly returning versions < 1.0.0.

Signed-off-by: dblock <dblock@amazon.com>

* Removed V_7_10_3 which has not been released as of time of the fork.

Signed-off-by: dblock <dblock@amazon.com>

* Update check for current version to get unreleased versions.

- no unreleased version if the current version is "1.0.0"
- add unit tests for OpenSearch 1.0.0 with legacy ES versions.
- update VersionUtils to include all legacy ES versions as released.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

Co-authored-by: Rabi Panda <adnapibar@gmail.com>
2021-06-02 16:03:50 -04:00
Nick Knize d821b15a6b
[BWC] fix mixedCluster and rolling upgrades (#775)
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: Shweta Thareja <tharejas@amazon.com>
2021-05-28 00:42:57 -05:00
ashwinpankaj 002ce9eddf
An allocation constraint mechanism, that de-prioritizes nodes from getting picked for allocation if they breach certain constraints (#680)
* An allocation constraint mechanism, that de-prioritizes nodes from getting picked for allocation if they breach certain constraints

Signed-off-by: Ashwin Pankaj <appankaj@amazon.com>
2021-05-26 12:42:37 +05:30
Himanshu Setia 6f893ed1cd
Merging javadoc feature branch changes to main (#715)
* Adds a gradle plugin to validate missing javadocs

Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fix Javadoc errors in module `client/rest` (#685)

* Fix Javadoc errors in client/rest module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Add package info file in client/rest module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Fix typos

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Add exception documentation to Javadoc

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Fixes precommit task configuration failures due to newly added missin… (#707)

* Fixes precommit task configuration failures due to newly added missingJavadoc task

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fixes javadoc task errors due to PR#685

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Updated CONTRIBUTING.md for info on javadocs

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Correcting licenses and naming

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Correcting version info

Signed-off-by: Himanshu Setia <setiah@amazon.com>

Co-authored-by: Gregor Zurowski <gregor@zurowski.org>
2021-05-18 13:21:41 -07:00
Ketan Verma ce12097095
Support Data Streams in OpenSearch (#690)
This commit adds support for data streams by adding a DataStreamFieldMapper, and making timestamp 
field name configurable. Backwards compatibility is supported.

Signed-off-by: Ketan Verma <ketan9495@gmail.com>
2021-05-17 09:09:25 -05:00
Rabi Panda 943c778a7f
[CVE-2018-11765] Upgrade hadoop dependencies for hdfs plugin (#654)
Hadoop 2.8.5 has been reported to have CVEs (https://bugzilla.redhat.com/show_bug.cgi?id=1883549). We need to upgrade this to 2.10.1. This also updates the hadoop-minicluster version to 2.10.1 as well. This upgrade also brings in two additional dependencies, woodstox-core and stax2-api that are added along with the sha1s, licenses and notices.

Also upgrade guava to the latest as per the CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-05-13 14:56:47 -07:00
Rabi Panda 67460c7804
Update hadoop-minicluster version for test fixture. (#645)
The `hadoop-minicluster:2.8.5` which is used for integration tests, has dependencies which have been flagged for having security vulnerabilities. Update this to the latest version `3.3.0`

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-05-04 10:48:43 -07:00
piyush 7ccb714045
Speedup snapshot stale indices delete (#613)
Instead of snapshot delete of stale indices being a single threaded operation this commit makes 
it a multithreaded operation and delete multiple stale indices in parallel using SNAPSHOT 
threadpool's workers.

Signed-off-by: Piyush Daftary <piyush.besu@gmail.com>
2021-04-27 13:28:35 -05:00
Nick Knize c5a3c3cb41
Update lucene version to 8.8.2 (#557)
This commit updates the codebase to the latest released version of Lucene.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-23 09:48:41 -05:00
Nick Knize 6e04778d0c
[Versioning] Fix Version.fromString logic for legacy version (#604)
This commit fixes the Version.fromString logic to identify legacy versions. It
also adds an optional "distribution" field to the MainRespose for OpenSearch
version 1.0.0+. Any preceeding versions that do not contain the distribution
label will be handeled as legacy versions appropriately.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-22 23:04:44 -05:00
Nick Knize 0ba0e7cc26
[Versioning] Rebase to OpenSearch version 1.0.0 (#555)
This commit rebases the versioning to OpenSearch 1.0.0

Co-authored-by: Rabi Panda <adnapibar@gmail.com>

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-15 17:06:47 -05:00
Nick Knize 4dde0f2a3b
[DOCKER] add apt update to test fixture krb5kdc (#565)
* [DOCKER] add apt update to test fixture krb5kdc

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-15 16:06:21 -05:00
Rabi Panda 2a3ce0bb75
Fix rename issues and failing repository-hdfs tests. (#518)
This commit fixes some partial rename issues and as a result fixes the failing secure repository-hdfs tests.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-04-09 17:51:27 -07:00
Nick Knize 9168f1fb43
[License] Add SPDX and OpenSearch Modification license header (#509)
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-09 14:28:18 -05:00
Rabi Panda 2dca3462f2
Fix stragglers from renaming to OpenSearch work. (#483)
This commit fixes more instances where we missed renaming to OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-04-05 11:51:20 -07:00
Rabi Panda 70cad29af3
Perform more renaming to OpenSearch. (#470)
This commit performs more renaming to OpenSearch which were missed in the earlier commits.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-30 11:05:50 -07:00
Rabi Panda 3460a8c213
Fix a few more renaming issues. (#464)
This commit fixes some more missed instances where we can perform the renaming to OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-26 12:05:16 -07:00
Rabi Panda c79bab46a0
Fix multiple failing server tests. (#453)
This commit fixes some renaming issues which as a result fixes multiple failing unit tests in the server module.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-23 15:13:09 -07:00
Himanshu Setia 28bbd94d6f
[Rename] Refactoring Elastic references in docker and kerberos builds (#428) (#438)
Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-03-22 09:20:14 -05:00
Rabi Panda 8bba6603da [Rename] Replace more instances of Elasticsearch with OpenSearch. (#432)
This commit replaces more replaceable instances of Elasticsearch with OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 7051167c83 [Rename] remaining elasticsearch pass 1 (#416)
This commit refactors instances of 'elasticsearch' with opensearch everywhere
except references to issues, and other places needed to test compatibility with
old elasticsearch clusters.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda 2830b3442f [Rename] Temporarily suppress thirdPartyAudit errors for precommit pass. (#421)
Currently the thirdPartyAudit task is failing for the test:framework module after renaming to OpenSearch. We have created an issue and temporarily suppressed the errors to unblock the precommit.

Issue: https://github.com/opensearch-project/OpenSearch/issues/420

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda eddfe6760d [Rename] Fix issues for gradle precommit task. (#418)
Fix miscellaneous issues identified during `gradle precommit`. These issues are the side effects of the renaming to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda d1e070c92b [Rename] Fix import issues in tests. (#414)
This commit fixes the import issues in already refactored test packages.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize a0fa1a625e [Rename] remaining ES prefixed classes (#406)
This commit refactors remaining ES classes to OpenSearch prefix throughout the
code base. All references are also refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda df11cc9de4 [Rename] Fix gradle build as part of the renaming process. (#397)
This commit fixes the currently broken gradle build resulted from the renaming work. It reverts a few dependencies and comments out the `opensearch_distibutions` task which is currently failing for some builds. We will address these separately in the future once we have a working build.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Harold Wang 236de25ffb Rename remaining org.elasticsearch.plugins (#401)
Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 914da95c48 [Rename] build.gradle in test module (#403)
This commit refactors the build.gradle file in the test module from o.e.test to
o.opensearch.test namespace.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Himanshu Setia 9a51fb2f3b Reverts the rename for test/fixtures/old-elasticsearch done in #370 (#402)
Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-03-21 20:56:34 -05:00
Himanshu Setia d1e275c4e9 Refactor test/logger-usage and test/external-modules as per OpenSearch naming conventions (#398)
Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 13f6d23e40 [Rename] Property and metadata keys with prefix es. (#389)
Rename all property and metadata keys with prefix 'es.' to 'opensearch.'.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize f75803e1aa [Rename] Second pass clean up of imports and variable names in server module (#382)
This commit cleans up imports, variable names, comments, and other misc usages
of ES with the new OpenSearch name.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Himanshu Setia b5e00e1e58 Refactors test/fixtures as per OpenSearch naming (#370)
Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-03-21 20:56:34 -05:00
Harold Wang 70e62f4609 [Rename] server/src/test/java/org/elasticsearch/test (#377)
* Refactor module server/src/test/java/org/elasticsearch/test

Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 2df59c4a4b [Rename] test framework remaining tests (#375)
This commit refactors the remaining tests in the test/framework module. All
other references throughout the codebase are also refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize 5b46a05702 [Rename] remaining packages and resources in test/fixture (#364)
This commit refactors the remaining o.e.index and o.e.test packages in the
test/fixtures module. References throughout the codebase are also refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize 5bef7cec87 [Rename] env,geo,http,indices,ingest,plugin,upgrade packages from test framework (#363)
This commit refactors the following test framework packages:

* o.e.env
* o.e.geo
* o.e.http
* o.e.indices
* o.e.ingest
* o.e.plugin
* o.e.upgrades

to the o.opensearch namespace. All references throughout the test codebase have
been refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize 12f3233fa5 [Rename] framework/test/o.e.action,bootstrap,cli,client,cluster,common (#361)
This commit refactors the following test packages from the o.e namespace:

* o.e.action
* o.e.bootstrap
* o.e.cli
* o.e.client
* o.e.cluster
* o.e.common

to the o.opensearch namespace. Any references throughout the codebase are also
refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda 0aa52bd2e8 [Rename] refactor the libs/x-content module. (#354)
Refactor the code in the `libs/x-content` module and any references to those in the entire code base. The refactoring is done as part of the renaming to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Harold Wang 182a86ea63 Rename org.elasticsearch.node to org.opensearch.node (#351)
Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 07ca47a256 [Rename] remaining classes in o.e (#352)
This commit refactors the remaining classes from o.e to the o.opensearch
namespace. All references throughout the codebase have been refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda 972d8ea920 [Rename] refactor the libs/core module. (#350)
Refactor the code in the `libs/core` module and any references to those in the entire code base. The refactoring is done as part of the renaming to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 50c3251d36 [Rename] o.e.common.settings (#336)
This commit refactors o.e.common.settings package to the
o.opensearch.common.setttings namespace. All references throughout the codebase
are refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Harold Wang 8ee7296fea [Rename] org.elasticsearch.plugins (#348)
* Rename org.elasticsearch.gateway to org.opensearch.gateway

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename org.elasticsearch.http to org.opensearch.http

Signed-off-by: Harold Wang <harowang@amazon.com>

* Renames org.elasticsearch.plugins to org.opensearch.plugins

Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 4e60df913f [Rename] refactor remaining of the `o.e.search` package in server module. (#345)
Refactor the remaining classes in the `org.elasticsearch.search` package in the server module, 

- Rename `org.elasticserach.search.aggregations` to `org.opensearch.search.aggregations`
- Rename instances of `org.elasticsearch.search` `org.opensearch.search`

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Harold Wang ac75d6aee2 [Rename] org.elasticsearch.http (#347)
* Rename org.elasticsearch.gateway to org.opensearch.gateway

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename org.elasticsearch.http to org.opensearch.http

Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Harold Wang 642f823228 Rename org.elasticsearch.gateway to org.opensearch.gateway (#346)
Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 3fc453eace [Rename] o.e.common.util (#337)
This commit refactors the o.e.common.util package to the
o.opensearch.common.util namespace. All references throughout the codebase have
been refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize b118646bef [Rename] o.e.common.xcontent (#339)
This commit refactors the o.e.common.xcontent package to
o.opensearch.common.xcontent. All references throughout the codebase have been
refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda e99f000366 [Rename] refactor o.e.search.aggregations.bucket package in the server module. (#326)
Rename `org.elasticserach.search.aggregations.bucket` to `org.opensearch.search.aggregations.bucket` in package names and references.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00