Commit Graph

2181 Commits

Author SHA1 Message Date
dependabot[bot] bce7f49944
Bump HdrHistogram from 2.1.9 to 2.1.12 in /server (#2135)
* Bump HdrHistogram from 2.1.9 to 2.1.12 in /server

Bumps [HdrHistogram](https://github.com/HdrHistogram/HdrHistogram) from 2.1.9 to 2.1.12.
- [Release notes](https://github.com/HdrHistogram/HdrHistogram/releases)
- [Commits](https://github.com/HdrHistogram/HdrHistogram/compare/HdrHistogram-2.1.9...HdrHistogram-2.1.12)

---
updated-dependencies:
- dependency-name: org.hdrhistogram:HdrHistogram
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update test to match new behavior of HdrHistogram

There was a [change in behavior in HdrHistogram][1] that causes the
tests to fail with the new version. The fix here is to add another value
in the test data set for the failing test, which has the result of
making both versions of HdrHistogram agree on the percentile values.

[1]: 5c7226c4ff

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Remove unnecessary exclusion on third party audit

Signed-off-by: Andrew Ross <andrross@amazon.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
2022-05-05 11:30:12 -05:00
Peng Huo 0bab4730b6
Correct the skip version, multi_terms aggregation is supported on 2.1 (#3072)
Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-04-25 17:07:14 -04:00
Peng Huo 03fbca3f50
Add new multi_term aggregation (#2687)
Adds a new multi_term aggregation. The current implementation focuses 
on adding new type aggregates. Performance (latency) is suboptimal in this
iteration, mainly because of brute force encoding/decoding a list of values
into bucket keys. A performance improvement change will be made as a 
follow on.

Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-04-21 09:06:33 -05:00
Suraj Singh dbdee30a37
[Type Removal] Remove TypeFieldMapper usage, remove support of `_type` in searches and from LeafFieldsLookup (#3016)
Removes TypeFieldMapper and _type support from searches

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-04-20 22:22:11 -05:00
Owais Kazi 3c5d997a76
Added a new line linter (#2875)
* Added linter to add new line

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed new lines

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Ignore empty files

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Updated DEVELOPER GUIDE

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Renamed workflow file

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed failing tests

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-04-13 14:14:18 -04:00
Tianli Feng 08e4a35839
Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Ingest APIs and Script APIs (#2682)
- Deprecate the request parameter `master_timeout` that used in Ingest APIs and Script APIs which have got the parameter. 
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-13 11:13:37 -07:00
Tianli Feng 455fad34ee
Change deprecation message for API parameter value 'master_node' of parameter 'metric' (#2880)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-12 14:58:19 -07:00
Tianli Feng a89b7e6216
Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Snapshot APIs (#2680)
* Deprecate the request parameter `master_timeout` that used in Snapshot APIs which have got the parameter. 
* Add alternative new request parameter `cluster_manager_timeout`.
* Add unit tests.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-11 20:42:21 -07:00
Tianli Feng ba3ed8a9cb
Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Index Template APIs (#2678)
- Deprecate the request parameter `master_timeout` that used in Index Template APIs which have got the parameter. 
(The other Index APIs are addressed in PR https://github.com/opensearch-project/OpenSearch/pull/2660) 
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-11 17:29:06 -07:00
Tianli Feng ba8657aca5
Change deprecation message for REST API parameter 'master_timeout' to specify the version of removal (#2863)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-11 15:59:04 -07:00
Tianli Feng 7aa496f9dd
Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Index APIs except index template APIs (#2660)
- Deprecate the request parameter `master_timeout` that used in Index APIs which have got the parameter. 
(except index template APIs, which is addressed in PR https://github.com/opensearch-project/OpenSearch/pull/2678) 
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-05 10:47:52 -07:00
Nick Knize 406ee36b15
Fix TaskInfo serialization and unmute task_list bwc test (#2766)
Fixes the version check around resourceStats serialization and unmutes the
failing bwc test.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-05 06:25:45 -05:00
Nicholas Walter Knize fbb2725ec1
[Mute] tasks.list/10_basic/tasks_list
Muting test until bwc fix is in place.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-04 18:06:29 -05:00
Tianli Feng 7d23b180f7
Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Cluster APIs (#2658)
- Deprecate the request parameter `master_timeout` that used in Cluster APIs which have got the parameter.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-04 13:47:34 -07:00
Tianli Feng 78465b4a08
Add request parameter 'cluster_manager_timeout' as the alternative for 'master_timeout', and deprecate 'master_timeout' - in CAT APIs (#2557)
Apply the change of CAT Nodes API in PR #2435 to other applicable CAT APIs.

- Deprecate the request parameter `master_timeout` that used in many CAT APIs.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-04-01 16:08:44 -07:00
Nick Knize 86eb24d27f
[Remove] types from rest-api-spec endpoints (#2689)
Removes types from rest-api-spec endpoints

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-31 23:08:19 -05:00
Tianli Feng e0f770643c
Add 'cluster_manager_node' into ClusterState Metric as an alternative to 'master_node' (#2415)
* Add `cluster_manager_node` into `ClusterState Metric`, as an alternative to `master_node`. So that the request parameter "metric" in `Cluster reroute` and `Cluster state` API accept the new value `cluster_manager_node`
* Deprecate the enum value of `Metric: MASTER_NODE("master_node")`
* Add YAML REST tests for the new parameter value for "cluster state" and "cluster reroute" API

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:09:37 -07:00
Tianli Feng ea31483126
Add a new node role 'cluster_manager' as the alternative for 'master' role and deprecate 'master' role (#2424)
Add a new node role `cluster_manager`, as the replacement for `master` role, that used in setting `node.roles: [ master ]`. 
They have got the same functionality, but can NOT be assigned to `node.roles` together.
* Add `CLUSTER_MANAGER_ROLE` in `DiscoveryNodeRole` class, and deprecate `MASTER_ROLE`
* Remove `MASTER_ROLE` from the `DiscoveryNodeRole.BUILT_IN_ROLES`, and temporarily load the role by the existing method `DiscoveryNode.setAdditionalRoles`
* Add a method `validateRole()` in `DiscoveryNodeRole` class, it's used to validate a specific role is compatible with the other roles to be assigned to a node together.
* Add deprecation message when assigning `master` role in setting `node.roles`
* Replace most `MASTER_ROLE` with `CLUSTER_MANAGER_ROLE` in current unit and integration tests
* Add new unit and integration tests to validate `CLUSTER_MANAGER_ROLE` and `MASTER_ROLE` can be treated as the same.

More explanation:
* New node will have "cluster_manager", "data", and "ingest" roles by default. Which means the default value of setting "node.roles" will be `["cluster_manager","data","ingest"]`, instead of `["master","data","ingest"]`
* "cluster_manager" role will be treated as "master" role in the OpenSearch node of previous versions.
* "cluster_manager” role and "master" role have got the same abbreviation name: "m"

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:06:06 -07:00
Tianli Feng 641350b0b6
Replace 'master' with 'cluster_manager' in 'GET Cat Nodes' API (#2441)
* Replace 'master' with 'cluster_manager' in the table header of [GET Cat Nodes](https://opensearch.org/docs/latest/opensearch/rest-api/cat/cat-nodes/) API, to promote inclusive language.
* Add `cm` as the alias for the table header `cluster_manager`
* Add `master` as the alias for the table header `cluster_manager`, for keeping compatibility when using `GET _cat/nodes?v&h=master` to show the specific column only.
* Add YAML rest test

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:04:49 -07:00
Tianli Feng ee048851f5
Replace 'discovered_master' with 'discovered_cluster_manager' in 'GET Cat Health' API (#2438)
* Replace 'discovered_master' with 'discovered_cluster_manager' in the table header of `GET Cat Health` API, to promote inclusive language.
* Add `dcm` as the alias for the table header `discovered_cluster_manager`
* Add `discovered_master` as the alias for the table header `discovered_cluster_manager`, for keeping compatibility when using `GET _cat/nodes?v&h=discovered_master` to show the specific column only.
* Add YAML rest test

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:04:03 -07:00
Tianli Feng 1c2bdd8642
Add a field discovered_cluster_manager in get cluster health api (#2437)
* Add a new field discovered_cluster_manager to the response of `GET _cluster/health` API, which has got the same value with the existing discovered_master field, aims to replace the non-inclusive term "master" in the API response field.
* Add yaml rest test to validate `discovered_cluster_manager` and `discovered_master` field have got the same value.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:03:06 -07:00
Tianli Feng a87c9d4455
Add request parameter 'cluster_manager_timeout' as the alternative for 'master_timeout', and deprecate 'master_timeout' - in CAT Nodes API (#2435)
* `master_timeout` is being used in multiple APIs, such as cat allocation and cat nodes APIs. The PR take CAT Nodes API as a sample.
* Add a new request parameter `cluster_manager_timeout`, as the alternative for the existing `master_timeout`, in CAT Nodes API
* Throw an exception when both master_timeout and cluster_manager_timeout are assigned but with different values, such as `GET _cat/nodes?v&master_timeout=1s&cluster_manager_timeout=2s`. This is addressed by the temporary method `validateParamValuesAreEqual()` added in `RestRequest` class.
* Add deprecation log when using request parameter `master_timeout`
* Deprecate request parameter `master_timeout` of in rest api spec
* Add unit tests

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:02:06 -07:00
Tianli Feng 3da9eb642c
Add a new REST API endpoint 'GET _cat/cluster_manager' as the replacement of 'GET _cat/master' (#2404)
* Add a new path `/_cat/cluster_manager` for the REST request handler `RestMasterAction`
* Deprecate the existing path `/_cat/master`
* Change the name of the `RestMasterAction` handler from `cat_master_action` to `cat_cluster_manager_action`
* Change the response of `GET _cat` to only show the new path `/_cat/cluster_manager`
* Rename the JSON rest-api-spec file `cat.master.json` to `cat.cluster_manager.json` and update the paths.
* Add YAML REST test. By default, it will run test for both endpoints `GET _cat/cluster_manager` and `GET _cat/master`

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 15:00:23 -07:00
Nick Knize 05a5819243
[Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30 (#2487)
* [Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30

Upgrades from Lucene 9.0.0 to 9.1.0-snapshot-ea989fe8f30 in preparation for
9.1.0 GA.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Add spanishplural token filter

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix KNOWN_TOKENIZERS

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-16 15:47:25 -04:00
Nick Knize 757abdb9a0
[Refactor] LuceneChangesSnapshot to use accurate ops history (#2452)
Improves the LuceneChangesSnapshot to get an accurate count of recovery
operations using sort by sequence number optimization.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-15 13:13:54 -05:00
Suraj Singh 2b68b14629
[Remove] Type from TermsLookUp (#2459)
* [Remove] Type from TermsLookUp

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Fix unit test failure

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-03-14 19:09:04 -07:00
Nick Knize 95d4750249
[Remove] types from translog (#2439)
Removes persisting the type in the translog since types are no longer supported.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-12 14:03:00 -06:00
Nick Knize b00b3ce2b5
[Remove] Multiple Types from IndexTemplateMetadata (#2400)
Removes multi-type support from IndexTemplateMetadata so that instead of holding
a map of multiple types to mappings, it only returns a single mapping for a
single type. Also removes type from documentMapper() method to avoid any
accidental NullPointerExceptions in the internal mapping retrieval.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-10 22:36:08 -06:00
Suraj Singh a6a47e7321
Remove inclue_type_name parameter from rest api spec (#2410)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-03-08 14:43:04 -08:00
Nick Knize c3712a51b3
[Remove] include_type_name from HLRC (#2397)
Removes include_type_name from the high level reset client along with relevant
deprecated methods in IndicesClient. All tests are updated to remove the
parameter from the rest requests along with various toXContent methods that are
no longer required.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-08 11:30:27 -06:00
Suraj Singh 3d5aff4b91
Remove type end-points from search and related APIs (#2263)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-03-02 13:21:00 -08:00
Nick Knize 897f4e7295
[Remove] deprecated getMapping API from IndicesClient (#2262)
Removes the deprecated types based get, getMapping, getAsync, and
getMappingAsync methods from IndicesClient. It also removes extra nesting of
mappings belong the deprecated type named object and removes the types based
methods from the affected request classes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-02 13:44:04 -06:00
Peng Huo 9e225dc9b8
Fix flaky test case - string profiler via global ordinals (#2226)
forcemerge to one segment before executing aggregation query.

Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-03-02 12:34:02 -06:00
Matt Weber 9f6e2e56e0
Update bwc verions for (#2237) (#2248)
Update the bwc versions for #2237 since it was backported to 1.3.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-02-24 16:14:49 -08:00
Nick Knize d892c51d66
[Remove] Types from DocWrite Request and Response (#2239)
Removes type support from DocWrite Request and Response, all derived classes,
and all places used.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-24 15:57:44 -06:00
Matt Weber 37235fafd9
Case Insensitive Support in Regexp Interval (#2237)
Add a `case_insensitive` flag to regexp interval source.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-02-24 14:54:13 -06:00
Suraj Singh cbfcad9054
Removes type mappings from mapping APIs (#2238)
Removes type mappings from GetMapping, GetFieldMapping and PutMapping APIs

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-24 11:23:36 -08:00
Nick Knize 3445befd36
[Remove] types from SearchHit and Explain API (#2205)
* [Remove] types from SearchHit and Explain API

Removes type support from SearchHit and Explain API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix failing tests

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* move TYPES_DEPRECATION_MESSAGE from RestExplainAction to FullClusterRestartIT

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix FullClusterRestartIT

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix failing tests

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix testSoftDelete

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-23 11:40:30 -08:00
Nick Knize 1b571ece28
[Remove] type support from Bulk API (#2215)
Removes all support for type parameter from the Bulk API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-23 11:53:27 -06:00
Nick Knize 3e9a420533
[Remove] 6x skip from yml (#2153)
Remove all 6x skip versions from yaml tests since mix 2.0 and 6x clusters are
not supported.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-22 23:20:18 -06:00
Suraj Singh 52eaf96a88
[Remove] Type mapping parameter from document update API (#2204)
* Remove type param from yml and integ test files

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Remove type mapping specific update API

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-22 11:54:03 -08:00
Nick Knize 2e3f410957
[Remove] Types from GET/MGET (#2168)
Removes type support from the get and mget transport and rest actions along
with removal from ShardGetService.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-21 11:37:27 -06:00
Matt Weber fc22350208
Update #2103 BWC Versions (#2173)
Change BWC checks to 1.3 since #2103 was backported.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-02-18 12:15:11 -06:00
Nick Knize 7fe642fda5
[Remove] Type from Search Internals (#2109)
With types deprecation the type support is removed from internal search API
(SearchRequest and QueryShardContext).

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-17 16:17:31 -06:00
Matt Weber cfd96e3e4b
Support unordered non-overlapping intervals (#2103)
* Support unordered non-overlapping intervals

This commit exposes Intervals.unorderedNoOverlaps (LUCENE-8828).

Signed-off-by: Matt Weber <matt@mattweber.org>

* Replace ordered and overlap flags with enum.

- add IntervalMode enum
- deprecate ordered
- bwc between IntervalMode and ordered flag
- update tests
- update rest tests

Signed-off-by: Matt Weber <matt@mattweber.org>

* fix annotated text plugin test

Signed-off-by: Matt Weber <matt@mattweber.org>

* bwc to 2x only

Signed-off-by: Matt Weber <matt@mattweber.org>

* BWC rest tests for ordered flag

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-02-17 17:10:22 -05:00
Peng Huo 9ea25c4f2e
Adjust main version after backport to 1.x (#2147)
Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-02-17 12:17:10 -06:00
Nick Knize c0c7d7aedb
[Remove] Type parameter from TermVectors API (#2104)
Remove document type support from the TermVectors API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-15 14:49:08 -06:00
Suraj Singh 892801a074
Remove type mapping from document index API (#2026)
* Remove doc type specific indexing APIs and relevant changes

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Remove type param from yml and integ test files

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-09 13:34:10 -06:00
Nick Knize 0791c88af1
[Remove] Deprecated Synced Flush API (#1761)
Remove the deprecated sync flush API which was replaced by sequence
number and retention lease mechanisms and no longer used in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 22:18:48 -06:00
Nick Knize 7058256e45
Enforce soft deletes (#1903)
This commit enforces soft deletes as required in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-18 12:54:42 -06:00