Commit Graph

55381 Commits

Author SHA1 Message Date
Owais Kazi 054595b140
Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation (#1983)
* Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed spotless imports

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-01-27 09:26:32 -08:00
Sergey Nuyanzin e399ac1f27
Add IssueNavigationLink (#1964)
Signed-off-by: Sergey Nuyanzin <snuyanzin@gmail.com>
2022-01-27 09:04:03 -08:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
Nick Knize 0791c88af1
[Remove] Deprecated Synced Flush API (#1761)
Remove the deprecated sync flush API which was replaced by sequence
number and retention lease mechanisms and no longer used in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 22:18:48 -06:00
Nick Knize 0f3b72b7bb
[Deprecate] Setting explicit version on analysis component (#1978)
Lucene 9 removes the ability to define an explicit version on an analysis 
component. The version parameter is deprecated at parse time and a warning is 
issued 
to the user through the deprecation logger.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:46:28 -06:00
Nick Knize a18b4e35b8
[Deprecate] index.merge.policy.max_merge_at_once_explicit (#1981)
max_merge_at_once_explicit is removed in lucene 9 so the index setting is
deprecated for removal in the next major release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:33:52 -06:00
Matt Weber 0bc30b4314
Wildcard max_expansion version check update (#1980)
Change from 2x to 1.3 for wildcard interval max_expansion version check
since #1916 was backported.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-26 18:43:20 -06:00
Peng Huo 7a9314a3bb
Support _first and _last parameter for missing bucket ordering in composite aggregation (#1942)
Adds support for _first and _last parameters for missing bucket ordering in composite aggregation. 
By default, if order is asc, missing_bucket at first, if order is desc, missing_bucket at last. If 
missing_order is "_first" or "_last", regardless order, missing_bucket is at first or last respectively.

Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-01-26 16:36:50 -06:00
Saurabh Singh 447b20c457
Delay the request size calculation until required by the indexing pressure framework (#1592)
Logically delay the request size calculation until it's absolutely required by the 
indexing pressure framework.

Signed-off-by: Saurabh Singh <sisurab@amazon.com>
2022-01-26 15:06:01 -06:00
Dave Lago 6bb560f7f1
Modified TESTING instructions to clarify use of testing classes (#1930)
Improve TESTING instructions to clarify use and purpose of testing classes

Signed-off-by: Dave Lago <davelago@amazon.com>
2022-01-26 14:54:58 -06:00
Andriy Redko 1018b9c8dc
Update Mockito to 4.3.1 (#1973)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-26 14:53:41 -06:00
Rishikesh Pasham 3f92aecddf
Enabling Sort Optimization to make use of Lucene (#1974)
Remove sort optimization. From Lucence 8.10.1, Numeric sort optimization is 
already done in Lucene Comparators. Lucene also supports sort optimization 
with search_after queries.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 14:36:07 -06:00
Matt Weber 3b8db91abe
Add max_expansions option to wildcard interval (#1916)
Add support for setting the max expansions on a wildcard interval.
The default value is still 128 and the max value is bounded by
`BooleanQuery.getMaxClauseCount()`.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-26 13:06:39 -06:00
Nick Knize 57ac788bb6
[Refactor] InternalEngine to always use soft deletes (#1933)
Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This commit refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 12:28:24 -06:00
ankitkala 56ae7fab63
Add hook to execute logic before Integ test task starts (#1969)
Signed-off-by: Ankit Kala <ankikala@amazon.com>
2022-01-25 14:47:45 +05:30
Vacha 3ce377f5cc
Using pull_request_target in place of pull_request (#1952)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-21 12:32:00 -08:00
Venkata Jyothsna Donapati 02c6a7f156
Prefer adaptive replica selection over awareness attribute based routing (#1107)
* Prefer adaptive replica selection over awareness attribute based routing

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor changes

Signed-off-by: jyothsna <donapv@amazon.com>

* Address review comments

Signed-off-by: jyothsna <donapv@amazon.com>

* Fix checkstyle

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor fix

Signed-off-by: jyothsna <donapv@amazon.com>
2022-01-21 11:11:49 -06:00
Sarat Vemulapalli 781156471a
Removing lingering transportclient (#1955)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-01-21 01:00:50 -08:00
Suraj Singh 27e05cc1dc
[BUG] Wait for outstanding requests to complete in LastSuccessfulSett… (#1939)
* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-20 17:26:53 -06:00
Ankit Jain 6eb8f6f307
Prioritize primary shard movement during shard allocation (#1445)
When some node or set of nodes is excluded (based on some cluster setting) 
BalancedShardsAllocator iterates over them in breadth first order picking 1 shard from 
each node and repeating the process until all shards are balanced. Since shards from 
each node are picked randomly it's possible the p and r of shard1 is relocated first 
leaving behind both p and r of shard2. If the excluded nodes were to go down the 
cluster becomes red. 

This commit introduces a new setting  "cluster.routing.allocation.move.primary_first" 
that prioritizes the p of both shard1 and shard2 first so the cluster does not become 
red if the excluded nodes were to go down before relocating other shards. Note that
with this setting enabled performance of this change is a direct function of number 
of indices, shards, replicas, and nodes. The larger the indices, replicas, and 
distribution scale, the slower the allocation becomes. This should be used with care.

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2022-01-20 17:02:54 -06:00
Tianli Feng 8b8d04173c
Update protobuf-java to 3.19.3 (#1945)
* Update protobuf-java to 3.19.3

Signed-off-by: Tianli Feng <ftl94@live.com>

* Exclude some API usage violations in the package com.google.protobuf for thirdPartyAudit task to pass

Signed-off-by: Tianli Feng <ftl94@live.com>
2022-01-20 11:05:28 -08:00
Vacha e0216dad0f
Using custom branch name for auto backporting PRs (#1862)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-19 16:23:43 -08:00
Andriy Redko 32f2189686
Update Netty to 4.1.73.Final (#1936)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-18 16:30:18 -08:00
Poojita Raj 4a8c51f15f
Add release notes for 1.2.4 (#1934)
Signed-off-by: Poojita Raj <poojiraj@amazon.com>
2022-01-18 16:23:11 -08:00
Nick Knize 7058256e45
Enforce soft deletes (#1903)
This commit enforces soft deletes as required in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-18 12:54:42 -06:00
Suraj Singh 7691f075cb
[Bug] Wait for outstanding requests to complete (#1925)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-18 10:49:47 -05:00
dependabot[bot] d3a4c47b69
Bump com.gradle.enterprise from 3.8 to 3.8.1 (#1920)
Bumps com.gradle.enterprise from 3.8 to 3.8.1.

---
updated-dependencies:
- dependency-name: com.gradle.enterprise
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-18 10:46:18 -05:00
Nick Knize 81d998dd7a
[BWC] Ensure 2.x compatibility with Legacy 7.10.x (#1902)
* [BWC] Ensure 2.x compatibility with Legacy 7.10.x

This commit fixes TransportHandshaker to send a spoofed Legacy 7.10.2 mincompat
version to ensure OpenSearch 2.x nodes can join a Legacy 7.10.x cluster for
rolling upgrade support. Without this change 7.10.x and OpenSearch 2.x mixed
cluster bwc tests would fail.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix AbstractSimpleTransportTestCase failures

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* change v3 to local static

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* make V_3_0_0 package private and reuse in test

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-17 12:26:28 -05:00
Andriy Redko dd8ad29770
Fixing org.opensearch.common.network.InetAddressesTests.testForStringIPv6WithScopeIdInput (#1913)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 15:23:33 -06:00
Matt Weber 524448fc41
Expand SearchPlugin javadocs. (#1909)
Add and clarify some search plugin point documentation.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 13:06:32 -06:00
Andriy Redko f059738aac
[BUG] Serialization bugs can cause node drops (#1885)
This commit restructures InboundHandler to ensure all data 
is consumed over the wire.

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 13:02:34 -06:00
Matt Weber e7d44c20e9
Make SortBuilders pluggable (#1856)
Add the ability for plugin authors to add custom sort builders.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 11:06:13 -06:00
Andriy Redko 6dcfe8cdcc
Fix o.o.transport.netty4.OpenSearchLoggingHandlerIT stack overflow test failure (#1900)
Attempt to fix o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow by 
hardening test expectation patterns in regex patterns

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 10:52:57 -06:00
Tyler 0f31d62cd9
File name correction to follow existing convention (#1874)
Updated the name of the file to follow existing naming conventions. I discovered this due to the 1.2.3 hyperlink on "/docs/1.2/version-history/" being broken.

Signed-off-by: Tyler McVaney <tylermcvaney@gmail.com>
2022-01-13 10:48:02 -05:00
Sarat Vemulapalli 292efe2040
Added help to build distributions in docs (#1898)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-01-13 10:01:54 -05:00
Owais Kazi cfc9ec292d
Linked the formatting setting file (#1860)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-01-13 10:00:32 -05:00
Nick Knize 0887ac6d92
[Remove] Old Translog Checkpoint Format (#1884)
This commit removes support for old translog checkpoint formats prior to 7.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-11 12:58:35 -06:00
dependabot[bot] c457272e6b
Bump com.gradle.enterprise from 3.5 to 3.8 (#1880)
Bumps com.gradle.enterprise from 3.5 to 3.8.

---
updated-dependencies:
- dependency-name: com.gradle.enterprise
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-11 10:15:43 -05:00
Rabi Panda 8fbd8cd998
Fix verifyVersions gradle task and cleanup bwcVersions (#1878)
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:51 -05:00
Rabi Panda 40442aae7f
Remove unwanted unreleased versions (#1877)
Only last major (2.0.0), last minor(1.3.0) and (1.2.4) can be unreleased

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:24 -05:00
Vacha 412bf82a4a
Updating dependabot open pr limits (#1875)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-11 09:18:53 -05:00
Rabi Panda 28f688fbc5
Use Version.compareMajor instead of using equals operator (#1876)
This test fails due to equality check of majors between OpenSearch Version 1.x and LegacyESVersion 7.x. They should be equivalent but in this case the check fails and LegacyVersion.v6.8.x is chosen which was removed in 52508d5.

This change fixes the comparison logic by using `Version.compareMajor` method instead.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 17:59:08 -08:00
Andriy Redko c192ecdfec
Attempt to fix :test:fixtures:s3-fixture:composeUp fails due to HTTP connection issue (#1866)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-10 17:34:52 -06:00
Andriy Redko 4dbf1d268c
Concurrent Searching (Experimental): modify profiling implementation to support concurrent data collection (#1673)
* Concurrent Searching (Experimental): modify profiling implementation to support concurrent data collection

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-10 17:11:48 -06:00
Rabi Panda ec24ee56e8
Remove deprecated search.remote settings (#1870)
This commit removes the deprecated search.remote.* settings.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 11:45:56 -06:00
Rabi Panda 52508d57f5
Remove LegacyESVersion.V_6_8_x constants (#1869)
This commit removes all usages of the `LegacyESVersion.V_6_8_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 11:43:15 -06:00
Andriy Redko c71ff87303
Fixing build failures after Flavor Serialization backport (#1867)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-07 16:43:56 -08:00
sharp-pixel 1bb8864178
Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions. (#1668)
* Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>

* Fix dangling HTML tags in javadoc to solve build issues with JDK 17.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>
2022-01-06 12:39:13 -05:00
Tianli Feng db23f72a2a
Update FIPS API libraries of Bouncy Castle (#1853)
* Update bc-fips to 1.0.2.1

Signed-off-by: Tianli Feng <ftl94@live.com>

* Update bcpg-fips to 1.0.5.1

Signed-off-by: Tianli Feng <ftl94@live.com>

* Update bctls-fips to 1.0.12.2

Signed-off-by: Tianli Feng <ftl94@live.com>

* Use the unified bouncycastle version for bcpkix-jdk15on in HDFS testing fixture

Signed-off-by: Tianli Feng <ftl94@live.com>
2022-01-05 16:59:55 -08:00
Vacha 6e26679274
Fixing typo in TESTING.md (#1849)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-05 08:17:26 -05:00