Refactors XContentType.fromMediaTypeOrFormat to fromMediaType so Accept headers
and Content-Type headers can be parsed separately. This helps in reusing the
same parse logic in for REST Versioning API support.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Replace internal usages of 'master' terminology in server/src/main directory
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Restore rename DISCOVERED_MASTER in ClusterHealthResponse
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Rename two methods in unit tests
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in ClusterState
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in LeaderChecker JoinHelper JoinTaskExecutor
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in more classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in more classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in more classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in more classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in more classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in more classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in DiscoveryNodes classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace master word in more classes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Correct mistakes
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Adjust format by spotlessApply task
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Change MASTER__NODE_BOOTSTRAPPED_MSG in test
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Fix SnapshotDisruptionIT by renaming to cluster-manager
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* [Remove] Type from nested fields using new metadata field mapper
types support is removed yet nested documents use the _type field to store the
path for nested documents. A new _nested_path metadata field mapper is added to
take the place of the _type field in order to remove the type dependency in
nested documents. BWC is handled in the new field mapper to ensure compatibility
with older versions.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* pr fixes
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* add test to merge same mapping with empty index settings
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* This change formalizes the notion of feature flags, and adds a "replication type" setting that will differentiate between document and segment replication, gated by a feature flag.
Since seg-rep is currently an incomplete implementation, the feature flag ensures that the setting is not visible to users without explicitly setting a system property. We can then continue to merge seg-rep related changes from the feature branch to `main` safely hidden behind the feature flag gate.
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Update security policy for testing feature flags
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
A few places still referenced legacy ESTestCase name. This refactors those
instances to OpenSearchTestCase.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
OpenSearch 2.0.0 no longer needs HLRC compatibility with legacy clients. This
commit removes all logic to spoof the version as a legacy cluster.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Adds a new multi_term aggregation. The current implementation focuses
on adding new type aggregates. Performance (latency) is suboptimal in this
iteration, mainly because of brute force encoding/decoding a list of values
into bucket keys. A performance improvement change will be made as a
follow on.
Signed-off-by: Peng Huo <penghuo@gmail.com>
* Refactoring GatedAutoCloseable to AutoCloseableRefCounted
This is a part of the process of merging our feature branch - feature/segment-replication - back into main by re-PRing our changes from the feature branch.
GatedAutoCloseable currently wraps a subclass of RefCounted. Segment replication adds another subclass, but this also wraps RefCounted. Both subclasses have the same shutdown hook - decRef. This change makes the superclass less generic to increase code convergence.
The breakdown of the plan to merge segment-replication to main is detailed in #2355
Segment replication design proposal - #2229
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Minor refactoring in RecoveryState
This change makes two minor updates to RecoveryState -
1. The readRecoveryState API is removed because it can be replaced by an invocation of the constructor
2. The class members of the Timer inner class are changed to private, and accesses are only through the public APIs
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Update RecoveryTargetTests to test Timer subclasses deterministically
This change removes the use of RandomBoolean in testing the Timer classes and creates a dedicated unit test for each. The common test logic is shared via a private method.
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Move the RecoveryState.Timer class to a top-level class
This will eventually be reused across both replication use-cases - peer recovery and segment replication.
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Further update of timer tests in RecoveryTargetTests
Removes a non-deterministic code path around stopping the timer, and avoids assertThat (deprecated)
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Rename to ReplicationTimer
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Remove RecoveryTargetTests assert on a running timer
Trying to serialize and deserialize a running Timer instance, and then checking for equality leads to flaky test failures when the ser/deser takes time.
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
AllFieldMapper was deprecated in legacy 6x. The remaining references are
removed, along with the field mapper and corresponding tests.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Adding a null pointer check to fix index_prefix query
Signed-off-by: Vacha Shah <vachshah@amazon.com>
* Adding test
Signed-off-by: Vacha Shah <vachshah@amazon.com>
- Deprecate the request parameter `master_timeout` that used in Ingest APIs and Script APIs which have got the parameter.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
This change refactors an earlier change to impose a reg-ex size limit on the include/exclude string. Instead of accepting an IndexSettings instance, the class now accepts a integer limit value. This is necessary because the IncludeExclude class is used outside the core codebase, whose use-cases may be unaware of indices and their settings. To ensure that a limit is always imposed, a default limit is defined in the class.
(cherry picked from commit ba1966853e728b153e42be59ba449420e79b09ee)
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
Co-authored-by: Kartik Ganesh <gkart@amazon.com>
* Deprecate the request parameter `master_timeout` that used in Snapshot APIs which have got the parameter.
* Add alternative new request parameter `cluster_manager_timeout`.
* Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
- Deprecate the request parameter `master_timeout` that used in Index Template APIs which have got the parameter.
(The other Index APIs are addressed in PR https://github.com/opensearch-project/OpenSearch/pull/2660)
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Updates to the large string reg-ex check
Removed the null-case for IndexSettings since this only occurs in tests. The tests now use a dummy Index Setting.
This change also fixes a bug with the base case handling of max regex length in the check.
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Bugfix to guard against stack overflow errors caused by very large reg-ex input
This change fixes a code path that did not properly impose the index-level max_regex_length limit. Therefore, it was possibly to provide ar arbitrarily large string as the include/exclude reg-ex value under search aggregations. This exposed the underlying node to crashes from a StackOverflowError, due to how the Lucene RegExp class processes strings using stack frames.
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Adding integration tests for large string RegEx
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Spotless
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Fix issue that deprecated setting 'cluster.initial_master_nodes' is not identified during node bootstrap
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Restore a variable name
Signed-off-by: Tianli Feng <ftianli@amazon.com>
- Deprecate the request parameter `master_timeout` that used in Index APIs which have got the parameter.
(except index template APIs, which is addressed in PR https://github.com/opensearch-project/OpenSearch/pull/2678)
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
- Deprecate the request parameter `master_timeout` that used in Cluster APIs which have got the parameter.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Changes version comparison logic to only translate major version when
comparing with legacy 7x versions. This is needed beginning in 2.0 so
that when running 2.0+ versions in bwc mode for 3.0+ upgrades, node
versions no longer have to translate major version or spoof to legacy
versions.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Apply the change of CAT Nodes API in PR #2435 to other applicable CAT APIs.
- Deprecate the request parameter `master_timeout` that used in many CAT APIs.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Move parseDeprecatedMasterTimeoutParameter method into BaseRestHandler class to reduce duplication
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Add more comments to unit test
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Make log message key different
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Prohibit using 'master_timeout' and 'cluster_manager_timeout' parameter together
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Add separate unit tests for BaseRestHandler.parseDeprecatedMasterTimeoutParameter()
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Restore unit test for cat allocation api
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Adjust format by spotlessApply task
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Fix testBothParamsNotValid() by adding warning assertion
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Enables merge on refresh and merge on commit in Opensearch by
way of two new index options:
index.merge_on_flush.max_full_flush_merge_wait_time and
index.merge_on_flush.enabled. Default merge_on_flush is disabled and
wait time is 10s.
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Change InboundDecoder ensureVersionCompatibility check for onOrAfter
V_2_0_0 instead of explicit version check. This way bug fix and minor
versions will correctly handshake in a mixed 1.x Cluster.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Tianli Feng
Nick Knize
Kartik Ganesh
Daniel Doubrovkine (dB.)
Peng Huo
Tushar Kharbanda
Suraj Singh
Wenjun Ruan
Andriy Redko
Ankit Jain
Vacha Shah
Poojita Raj
Owais Kazi
opensearch-trigger-bot[bot]
Suraj Singh
Mohit Godwani
Sruti Parthiban