Commit Graph

6074 Commits

Author SHA1 Message Date
Suraj Singh eb847aeeef
[Type removal] Remove type from BulkRequestParser (#3423)
* [Type removal] Remove type handling in bulk request parser

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* [Type removal] Remove testTypesStillParsedForBulkMonitoring as it is no longer present in codebase

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-05-23 17:01:22 -04:00
Poojita Raj a023ad9cba
[segment replication]Introducing common Replication interfaces for segment replication and recovery code paths (#3234)
* RecoveryState inherits from ReplicationState + RecoveryTarget inherits from ReplicationTarget

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* Refactoring: mixedClusterVersion error fix + move Stage to ReplicationState

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* pull ReplicationListener into a top level class + add javadocs + address review comments

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* fix javadoc

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* review changes

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* Refactoring the hierarchy relationship between repl and recovery

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* style fix

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* move package common under replication

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* rename to replication

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* rename and doc changes

Signed-off-by: Poojita Raj <poojiraj@amazon.com>
2022-05-23 12:19:54 -07:00
Nick Knize cb6148eb0d
Bump version 2.1 to Lucene 9.2 after upgrade (#3424)
Bumps Version.V_2_1_0 lucene version to 9.2 after backporting upgrage.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-23 10:28:13 -05:00
Nick Knize 5358502107
[Upgrade] Lucene-9.2.0-snapshot-ba8c3a8 (#3416)
Upgrades to latest snapshot of lucene 9.2.0 in preparation for GA release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-20 16:29:51 -05:00
Andriy Redko 96b45ca103
[REMOVE] Cleanup deprecated thread pool types (FIXED_AUTO_QUEUE_SIZE) (#3369)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-05-19 17:20:26 -04:00
Tianli Feng 507f8ccdbd
Replace internal usages of 'master' term in 'server/src/internalClusterTest' directory (#2521)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-19 10:23:59 -07:00
Suraj Singh 7a73fb561e
[Type removal] Remove _type from _bulk yaml test, scripts, unused constants (#3372)
* [Type removal] Remove redundant _type deprecation checks in bulk request

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* [Type removal] bulk yaml tests validating deprecation on _type and removal from scripts

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-05-18 17:04:29 -04:00
Suraj Singh daed5c190a
[Type removal] Remove _type deprecation from script and conditional processor (#3239)
* [Type removal] Remove _type deprecation from script and conditional processor

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Spotless check apply

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-05-18 17:01:43 -04:00
Ankit Jain f8b102c565
Removing hard coded value of max concurrent shard requests (#3364)
Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2022-05-18 12:51:21 -05:00
Suraj Singh 892e9846b5
[Type removal] Remove redundant _type in pipeline simulate action (#3371)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-05-18 13:43:32 -04:00
Andriy Redko f021860a25
[REMOVE] Cleanup deprecated thread pool settings (AutoQueueAdjustingExecutorBuilder) (#3352)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-05-17 18:02:58 -07:00
Tianli Feng 8f8909fd54
Replace 'whitelist' terminology in Java API (#3350)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-17 13:34:05 -07:00
Tianli Feng 00c0bf2dd9
[TEST] Add back necessary tests for deprecated settings that are replaced during applying inclusive naming (#2825)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-17 10:37:52 -07:00
Andriy Redko c4705808e9
Remove deprecated methods from JodaCompatibleZonedDateTime which are called by scripts (#3346)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-05-16 15:38:17 -07:00
Andriy Redko 38fb1d9275
Add resizable search queue to OpenSearch (picking up #826) (#3207)
* Make search/write queue resizable

Signed-off-by: Ruizhen <ruizhen@amazon.com>

* Address PR comments

Signed-off-by: Ruizhen <ruizhen@amazon.com>

* Refactoring resizable queue implementation

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Co-authored-by: Ruizhen <ruizhen@amazon.com>
2022-05-16 08:37:41 -04:00
Nick Knize 3aef125d0d
[Javadocs] add remaining internal classes and reenable missingJavadoc on server (#3296)
Adds the remaining javadocs to internal classes and reenables the missingJavadoc
gradle task on the server module. From here forward if class level javadocs are
missing in the server module, gradle check will fail!

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-12 14:29:58 -05:00
Tianli Feng 10bff0c9f5
Replace internal usages of 'master' term in 'test' directory (#3283)
* Replace internal usages of 'master' terminology in server/src/internalClusterTest directory

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace 'master' terminology with 'cluster manager' in test directory

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace 'master' terminology with 'cluster manager' in server/src/internalClusterTest directory

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Adjust format by spotlessApply task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace 'master' terminology with 'cluster manager' in test directory

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Adjust format by spotlessApply task

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-11 19:34:57 -04:00
Andriy Redko 677915dc00
Bootstrap should implement a denylist of Java versions (ranges) (#3164)
* Bootstrap should implement a denylist of Java versions (ranges)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-05-11 15:33:48 -07:00
Nick Knize ad7ce4cd44
[Javadocs] add to o.o.common (#3289)
Add javadocs to top level and internal classes in the org.opensearch.common
package.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-11 15:58:10 -05:00
Nick Knize 86f8dd65a4
[Javadocs] add to internal classes in o.o.http, indices, and search (#3288)
Adds javadocs to internal classes in org.opensearch.http, indices, and search
packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-11 13:52:28 -04:00
Nick Knize b78176afef
[Javadocs] add to o.o.action, index, and transport (#3277)
Adds javadocs to internal classes in org.opensearch.action, index, and transport
packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-10 17:18:07 -05:00
Nick Knize 3b71ea6c1e
[Javadocs] add to o.o.search.rescore,searchafter,slice, sort, and suggest (#3264)
Adds class level javadocs to org.opensearch.search, and
org.opensearch.search.rescore, searchAfter, slice, sort, and suggest
subpackages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-09 16:20:56 -07:00
Nick Knize 1118dcf372
[Javadocs] add to o.o.dfs,fetch,internal,lookup,profile, and query packages (#3261)
Adds class level javadocs to org.opensearch.dfs, fetch, internal, lookup,
profile, and query packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-09 14:05:59 -05:00
Nick Knize 625623f932
[Javadocs] add to o.o.search.aggs, builder, and collapse packages (#3254)
Adds class level javadocs to org.opensearch.search.aggregations, builder, and
collapse packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-09 11:20:19 -07:00
Nick Knize 15161a8649
[Javadocs] add to o.o.rest, snapshots, and tasks packages (#3219)
Adds javadocs to org.opensearch.rest, snapshots, and tasks packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-06 11:03:01 -07:00
Nick Knize a7715ab6ce
[Javadocs] add to o.o.transport (#3220)
Adds javadocs to the org.opensearch.transport package.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-06 08:36:04 -05:00
Nick Knize 7b75fb425f
[Javadocs] add to o.o.index and indices (#3209)
Adds javadocs to org.opensearch.index and indices packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-05 17:06:48 -05:00
opensearch-trigger-bot[bot] c41b41140b
Added bwc version 1.3.3 (#3205)
Signed-off-by: GitHub <noreply@github.com>

Co-authored-by: opensearch-ci-bot <opensearch-ci-bot@users.noreply.github.com>
2022-05-05 16:19:04 -04:00
Kartik Ganesh 44573baff2
Moving RecoveryState.Index to a top-level class and renaming (#3075)
* Moving RecoveryState.Index to a top-level class and renaming

This class is a building block of replication and will be re-used between peer recovery and segment replication. Thus, the inner class has been extracted to a top-level class and moved to the replication.common package. It has been renamed to ReplicationLuceneIndex to better reflect what it represents. It has two dependent inner classes from RecoveryState that have also been moved along with it - these remain inner classes since they are not currently used anywhere else. The RecoveryFilesDetails class has been renamed to FilesDetails and the FileDetail class has been renamed to FileMetadata.

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Incorporate PR comments

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Revert Project_Default.xml

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Update REST Action test to no longer mock a final class

Instead, the test now populates dummy data.

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Removing mocking of ReplicationLuceneIndex in RecoverySourceHandlerTests

The class has been marked final, so it can no longer be mocked. Instead, the test class sets up the lucene index class by adding the smae file metadata that is set up for the store.

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Configure @opensearch.internal as custom Javadoc tag

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Revert "Configure @opensearch.internal as custom Javadoc tag"

This reverts commit 2077d76e421fc2ceea7c1c9714b6a1906a555a31.

Signed-off-by: Kartik Ganesh <gkart@amazon.com>
2022-05-05 12:22:47 -07:00
dependabot[bot] bce7f49944
Bump HdrHistogram from 2.1.9 to 2.1.12 in /server (#2135)
* Bump HdrHistogram from 2.1.9 to 2.1.12 in /server

Bumps [HdrHistogram](https://github.com/HdrHistogram/HdrHistogram) from 2.1.9 to 2.1.12.
- [Release notes](https://github.com/HdrHistogram/HdrHistogram/releases)
- [Commits](https://github.com/HdrHistogram/HdrHistogram/compare/HdrHistogram-2.1.9...HdrHistogram-2.1.12)

---
updated-dependencies:
- dependency-name: org.hdrhistogram:HdrHistogram
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update test to match new behavior of HdrHistogram

There was a [change in behavior in HdrHistogram][1] that causes the
tests to fail with the new version. The fix here is to add another value
in the test data set for the failing test, which has the result of
making both versions of HdrHistogram agree on the percentile values.

[1]: 5c7226c4ff

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Remove unnecessary exclusion on third party audit

Signed-off-by: Andrew Ross <andrross@amazon.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
2022-05-05 11:30:12 -05:00
Nick Knize 741445a574
[Remove] TypeFieldMapper (#3196)
Removes last trace of TypeFieldMapper since types have been removed and
NestedPathFieldMapper is now available.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-05 09:35:41 -05:00
Nick Knize 14704b30ca
[Javadocs] add to o.o.monitor,persistance,plugins,repo,script,threadpool,usage,watcher (#3186)
Adds javadocs to classes in the org.opensearch.monitor, persistence, plugins,
repository, script, threadpool, usage, and watcher packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-04 15:21:35 -07:00
Nick Knize 4d3da00c12
[Javadocs] Add to o.o.disovery, env, gateway, http, ingest, lucene and node pkgs (#3185)
Adds javadocs to classes in the org.opensearch.discovery, env, gateway, http,
ingest, lucene, and node packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-04 14:32:01 -05:00
Nick Knize 6ca3278f96
[Javadocs] Add to remaining o.o.action classes (#3182)
Adds javadocs to remaining undocumented classes in o.o.action package.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-04 13:35:57 -05:00
Nick Knize 40024fcf71
[Javadocs] add to o.o.cluster (#3170)
Adds javadocs to classes in the o.o.cluster package.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-03 22:18:43 -05:00
Nick Knize c76a4c9014
[Remove] ShrinkAction, ShardUpgradeRequest, UpgradeSettingsRequestBuilder (#3169)
Removes unused ShrinkAction, ShardUpgradeRequest, and
UpgradeSettingsRequestBuilder classes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-03 22:18:28 -05:00
Nick Knize ad1c8038b0
[Javadocs] add to o.o.action.admin (#3155)
Adds javadocs to classes in package o.o.action.admin.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-03 22:16:48 -05:00
Nick Knize c13b679aad
[Javadocs] add to o.o.bootstrap, cli, and client (#3163)
Adds javadocs to o.o.bootstrap, o.o.cli, and o.o.client packages.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-03 14:44:59 -05:00
Nick Knize 5f531b366e
[Remove] AliasesExistAction (#3149)
With the removal of Transport Client AliasesExistAction is no longer needed and
is removed.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-03 12:05:41 -04:00
Nick Knize 15f035b479
[Remove] TypesExist Action (#3139)
The types exist transport action can be removed now that the TransportClient has
been removed and types support has been removed.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-03 10:26:37 -04:00
Tianli Feng bb870f70c8
Rename ClusterBlock description 'no master' to 'no cluster-manager' (#3133)
* Rename ClusterBlock description 'no master' to 'no cluster-manager'

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Adjust format by spotlessApply task

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-03 10:24:37 -04:00
Tianli Feng f58d98d261
Rename reserved node id '_must_join_elected_master_' to '_must_join_elected_cluster_manager_' that used by in DetachClusterCommand (#3116)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-02 20:23:11 -07:00
Nick Knize 21468279b2
[Javadoc] Add missing package-info.java files to server (#3128)
In preparation for re-enabling the missingJavadoc gradle task this change adds
in the missing package-info.java files to the server folder. For now general
javadocs are added to these files with the intent to clean up with better
descriptions over time.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-02 14:30:53 -05:00
Tianli Feng 017773c62d
Rename BecomeMasterTask to BecomeClusterManagerTask in JoinTaskExecutor (#3099)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-28 13:57:49 -04:00
Nick Knize d86c88fe59
[Refactor] XContentType to parse Accept or Content-Type headers (#3077)
Refactors XContentType.fromMediaTypeOrFormat to fromMediaType so Accept headers
and Content-Type headers can be parsed separately. This helps in reusing the
same parse logic in for REST Versioning API support.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-28 09:58:26 -05:00
Tianli Feng 79eb3b0492
Replace internal usages of 'master' term in 'server/src/main' directory (#2519)
* Replace internal usages of 'master' terminology in server/src/main directory

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Restore rename DISCOVERED_MASTER in ClusterHealthResponse

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Rename two methods in unit tests

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in ClusterState

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in LeaderChecker JoinHelper JoinTaskExecutor

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in more classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in more classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in more classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in more classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in more classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in more classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in DiscoveryNodes classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master word in more classes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Correct mistakes

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Adjust format by spotlessApply task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Change MASTER__NODE_BOOTSTRAPPED_MSG in test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Fix SnapshotDisruptionIT by renaming to cluster-manager

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-27 09:07:35 -04:00
Nick Knize 6b641d2fd2
[Remove] Type from nested fields using new metadata field mapper (#3004)
* [Remove] Type from nested fields using new metadata field mapper

types support is removed yet nested documents use the _type field to store the
path for nested documents. A new _nested_path metadata field mapper is added to
take the place of the _type field in order to remove the type dependency in
nested documents. BWC is handled in the new field mapper to ensure compatibility
with older versions.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* pr fixes

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* add test to merge same mapping with empty index settings

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-26 16:20:10 -04:00
Kartik Ganesh ee7b731975
Adds the replication type index setting, alongside a formal notion of feature flags (#3037)
* This change formalizes the notion of feature flags, and adds a "replication type" setting that will differentiate between document and segment replication, gated by a feature flag. 

Since seg-rep is currently an incomplete implementation, the feature flag ensures that the setting is not visible to users without explicitly setting a system property. We can then continue to merge seg-rep related changes from the feature branch to `main` safely hidden behind the feature flag gate.

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Update security policy for testing feature flags

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-25 10:51:02 -07:00
Daniel Doubrovkine (dB.) a389d1e221
Revert "Support task resource tracking in OpenSearch (#2639)" (#3046)
This reverts commit 6517eeca50.

Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-25 11:08:38 -05:00
Nick Knize 88bc268e29
[Rename] ESTestCase stragglers to OpenSearchTestCase (#3053)
A few places still referenced legacy ESTestCase name. This refactors those
instances to OpenSearchTestCase.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-23 16:50:55 -05:00