5878 Commits

Author SHA1 Message Date
Venkata Jyothsna Donapati
02c6a7f156
Prefer adaptive replica selection over awareness attribute based routing (#1107)
* Prefer adaptive replica selection over awareness attribute based routing

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor changes

Signed-off-by: jyothsna <donapv@amazon.com>

* Address review comments

Signed-off-by: jyothsna <donapv@amazon.com>

* Fix checkstyle

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor fix

Signed-off-by: jyothsna <donapv@amazon.com>
2022-01-21 11:11:49 -06:00
Sarat Vemulapalli
781156471a
Removing lingering transportclient (#1955)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-01-21 01:00:50 -08:00
Suraj Singh
27e05cc1dc
[BUG] Wait for outstanding requests to complete in LastSuccessfulSett… (#1939)
* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-20 17:26:53 -06:00
Ankit Jain
6eb8f6f307
Prioritize primary shard movement during shard allocation (#1445)
When some node or set of nodes is excluded (based on some cluster setting) 
BalancedShardsAllocator iterates over them in breadth first order picking 1 shard from 
each node and repeating the process until all shards are balanced. Since shards from 
each node are picked randomly it's possible the p and r of shard1 is relocated first 
leaving behind both p and r of shard2. If the excluded nodes were to go down the 
cluster becomes red. 

This commit introduces a new setting  "cluster.routing.allocation.move.primary_first" 
that prioritizes the p of both shard1 and shard2 first so the cluster does not become 
red if the excluded nodes were to go down before relocating other shards. Note that
with this setting enabled performance of this change is a direct function of number 
of indices, shards, replicas, and nodes. The larger the indices, replicas, and 
distribution scale, the slower the allocation becomes. This should be used with care.

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2022-01-20 17:02:54 -06:00
Nick Knize
7058256e45
Enforce soft deletes (#1903)
This commit enforces soft deletes as required in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-18 12:54:42 -06:00
Suraj Singh
7691f075cb
[Bug] Wait for outstanding requests to complete (#1925)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-18 10:49:47 -05:00
Nick Knize
81d998dd7a
[BWC] Ensure 2.x compatibility with Legacy 7.10.x (#1902)
* [BWC] Ensure 2.x compatibility with Legacy 7.10.x

This commit fixes TransportHandshaker to send a spoofed Legacy 7.10.2 mincompat
version to ensure OpenSearch 2.x nodes can join a Legacy 7.10.x cluster for
rolling upgrade support. Without this change 7.10.x and OpenSearch 2.x mixed
cluster bwc tests would fail.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix AbstractSimpleTransportTestCase failures

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* change v3 to local static

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* make V_3_0_0 package private and reuse in test

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-17 12:26:28 -05:00
Andriy Redko
dd8ad29770
Fixing org.opensearch.common.network.InetAddressesTests.testForStringIPv6WithScopeIdInput (#1913)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 15:23:33 -06:00
Matt Weber
524448fc41
Expand SearchPlugin javadocs. (#1909)
Add and clarify some search plugin point documentation.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 13:06:32 -06:00
Andriy Redko
f059738aac
[BUG] Serialization bugs can cause node drops (#1885)
This commit restructures InboundHandler to ensure all data 
is consumed over the wire.

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 13:02:34 -06:00
Matt Weber
e7d44c20e9
Make SortBuilders pluggable (#1856)
Add the ability for plugin authors to add custom sort builders.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 11:06:13 -06:00
Nick Knize
0887ac6d92
[Remove] Old Translog Checkpoint Format (#1884)
This commit removes support for old translog checkpoint formats prior to 7.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-11 12:58:35 -06:00
Rabi Panda
40442aae7f
Remove unwanted unreleased versions (#1877)
Only last major (2.0.0), last minor(1.3.0) and (1.2.4) can be unreleased

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:24 -05:00
Rabi Panda
28f688fbc5
Use Version.compareMajor instead of using equals operator (#1876)
This test fails due to equality check of majors between OpenSearch Version 1.x and LegacyESVersion 7.x. They should be equivalent but in this case the check fails and LegacyVersion.v6.8.x is chosen which was removed in 52508d5.

This change fixes the comparison logic by using `Version.compareMajor` method instead.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 17:59:08 -08:00
Andriy Redko
4dbf1d268c
Concurrent Searching (Experimental): modify profiling implementation to support concurrent data collection (#1673)
* Concurrent Searching (Experimental): modify profiling implementation to support concurrent data collection

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-10 17:11:48 -06:00
Rabi Panda
ec24ee56e8
Remove deprecated search.remote settings (#1870)
This commit removes the deprecated search.remote.* settings.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 11:45:56 -06:00
Rabi Panda
52508d57f5
Remove LegacyESVersion.V_6_8_x constants (#1869)
This commit removes all usages of the `LegacyESVersion.V_6_8_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 11:43:15 -06:00
Andriy Redko
c71ff87303
Fixing build failures after Flavor Serialization backport (#1867)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-07 16:43:56 -08:00
sharp-pixel
1bb8864178
Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions. (#1668)
* Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>

* Fix dangling HTML tags in javadoc to solve build issues with JDK 17.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>
2022-01-06 12:39:13 -05:00
Andriy Redko
385b268bc0
Update Mockito to 4.2.x (#1830)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-03 12:00:45 -05:00
Rabi Panda
06cbc47136
Remove LegacyESVersion.V_6_7_x constants (#1807)
This commit removes all usages of the `LegacyESVersion.V_6_7_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-02 14:09:55 -08:00
Rabi Panda
5ccb22fb4b
Remove LegacyESVersion.V_6_6_x constants (#1804)
This commit removes all usages of the `LegacyESVersion.V_6_6_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-29 22:43:53 -08:00
Tianli Feng
c05e0d90bd
[TEST] Fix FsHealthServiceTest by increasing the timeout period before checking the FS health after restoring the FS status (#1813)
Signed-off-by: Tianli Feng <ftl94@live.com>
2021-12-28 20:50:59 -08:00
Andriy Redko
65804d25a6
Update to log4j 2.17.1 (#1820)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-28 17:06:42 -05:00
Rabi Panda
01e1b94e47
Remove LegacyESVersion.V_6_5_x constants (#1794)
* Remove v6.5.x constants

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

* Address code review comments.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

* Add back version assumption for testRemoteClusterSettingsUpgraded

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

* Remove unused import after rebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-28 11:52:51 -06:00
Rabi Panda
7c73ed8d4c
Remove deprecated transport client (#1781)
This commit removes all usage of transport client in OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-28 11:28:51 -06:00
Rabi Panda
24d0c9b818
Remove transport client from tests. (#1809)
Transport client has been deprecated. In this commit, we are removing all its usages from the tests.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-27 17:43:18 -06:00
Suraj Singh
b00b45b25d
[Tests] ClusterHealthIT:testHealthOnMasterFailover - Increase master node timeout (#1812)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2021-12-27 13:26:18 -06:00
Rabi Panda
8b076b81ee
Remove LegacyVersion.v6.4.x constants (#1787)
This commit removes the `LegacyVersion.v6.4.x` constants and their usages, since we no longer need to support those in OpenSearch 2.0 onwards.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-23 16:54:37 -08:00
Rabi Panda
186959a6f5
Add bwc version 1.2.4 (#1796)
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-23 10:05:30 -05:00
Andrew Ross
96d55966a2
Use try-with-resources with MockLogAppender (#1595)
I previously added a helper that started a MockLogAppender to ensure it
was never added to a Logger before it was started. I subsequently found
the opposite case in RolloverIT.java where the appender was stopped
before it was closed, therefore creating a race where a concurrently
running test in the same JVM could cause a logging failure. This seems
like a really easy mistake to make when writing a test or introduce when
refactoring a test. I've made a change to use try-with-resources to
ensure that proper setup and teardown is done. This should make it much
harder to introduce this particular test bug in the future.
Unfortunately, it did involve touching a lot of files. The changes here
are purely structural to leverage try-with-resources; no testing logic
has been changed.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-12-21 15:00:55 -06:00
Andriy Redko
ca27c8fd4f
Update to log4j 2.17.0 (#1771) 2021-12-18 09:36:59 -08:00
Ankit Jain
3ed3fca4ae
Fixing allocation filters to persist existing state on settings update (#1718)
* Fixing allocation filters to persist existing state on settings update

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>

* Adding test for filter settings update

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>

* Adding more tests and review comments

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>

* Adding assertion and unit test for operation type mismatch

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>

* Updating test names

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2021-12-18 09:06:05 -05:00
Nick Knize
3f210dcfce
Remove LegacyESVersion.V_6_3_x constants (#1691)
This commit removes LegacyESVersion.V_6_3_x constants including all
pre-release versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-17 16:24:50 -06:00
Daniel Doubrovkine (dB.)
bbb6e656c0
Add version 1.2.3. (#1760)
Signed-off-by: dblock <dblock@dblock.org>
2021-12-17 09:14:37 -08:00
Poojita Raj
4b97713a2f
Bridging the gap in network overhead measurement in the profiler (#1360)
* Bridging the gap in network overhead measurement in the profiler

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* add tests + remove total n/w time field

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* refactor network time into a class

Signed-off-by: Poojita Raj <poojiraj@amazon.com>

* gradle test fix

Signed-off-by: Poojita Raj <poojiraj@amazon.com>
2021-12-16 19:50:04 -08:00
Nick Knize
6cc462b92d
[Remove] various builder and mapping deprecations (#1752)
This commit removes the following deprecations and invalid tests:

* LegacyUpdateMappingIntegrationIT
* ShapeBuilder new z axis checks
* MatchPhrase new zeroTermsQuery checks
* TypeQuery utf8
* CompositeValuesSource format checks
* LegacyESVersionTests bumped to 6.8.15
* LegacyDynamicMapping, LegacySimilarity, LegacyMapperService tests

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-16 18:42:39 -06:00
Nick Knize
8c2d8f5431
[Remove] Remaining Flavor Serialization (#1751)
This commit removes unnecessary serialization of unused flavor variable in build
metadata from V_1_3_0+

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-16 17:31:25 -06:00
Nick Knize
ef44182731
[Remove] DynamicTemplate deprecations (#1742)
This commit removes legacy version checks in DynamicTemplate parsing that are no
longer valid in OpenSearch 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-16 12:52:25 -06:00
Nick Knize
5550f8d7e2
[Remove] Analyzer Deprecations (#1741)
This commit removes deprecated analyzer instantiation that is no longer
permitted in OpenSearch 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-16 12:51:49 -06:00
Megha Sai Kavikondala
22bfadf324
Adding a cancelled field to tell if a cancellable task is cancelled (#1732)
* Changes made by adding the Task indicator and the IllegalArgs.

Signed-off-by: meghasaik <kavmegha@amazon.com>

* Changes made by changing the String message for uncancellable task.

Signed-off-by: meghasaik <kavmegha@amazon.com>
2021-12-16 09:49:49 -05:00
Vacha
e66ea2c4f3
Avoid logging duplicate deprecation warnings multiple times (#1660)
* Avoid logging duplicate deprecation warnings multiple times

Signed-off-by: Vacha <vachshah@amazon.com>

* Fixes test failures

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding deprecation logger tests

Signed-off-by: Vacha <vachshah@amazon.com>

* Using ConcurrentHashMap keySet

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-12-15 15:26:44 -08:00
Sarat Vemulapalli
a632b86e4c
Adding 1.2.2 (#1731)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2021-12-15 10:30:06 -05:00
Andriy Redko
6db435412b
Upgrade to log4j 2.16.0 (#1721)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-14 07:34:45 -05:00
Andrew Ross
309649ce8a
Upgrade to logj4 2.15.0 (#1698)
Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-12-10 13:03:41 -08:00
Daniel Doubrovkine (dB.)
ec00b86859
Add version 1.2.1. (#1701)
Signed-off-by: dblock <dblock@dblock.org>
2021-12-10 14:47:52 -05:00
Tianli Feng
baa10b9b20
Fix unit test testFailsHealthOnHungIOBeyondHealthyTimeout() by incresing the max waiting time before assertion (#1692)
* Increase the max waiting time before checking the FS health after restoring the FS status in the test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Remove another duplicate assetion statement

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2021-12-10 12:25:57 -05:00
Sarat Vemulapalli
e0e6995c4a
Updating Log4j to 2.11.2 (#1696)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2021-12-10 08:03:45 -08:00
Nick Knize
538b40a0dc
Remove LegacyESVersion.V_6_2_x constants (#1686)
This commit removes LegacyESVersion.V_6_2_x constants including all
pre-release versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-09 16:08:00 -06:00
Nick Knize
821417b9ab
Remove LegacyESVersion.V_6_1_x constants (#1681)
This commit removes LegacyESVersion.V_6_1_x constants including all
pre-release versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-08 23:11:36 -06:00