Commit Graph

55365 Commits

Author SHA1 Message Date
Venkata Jyothsna Donapati 02c6a7f156
Prefer adaptive replica selection over awareness attribute based routing (#1107)
* Prefer adaptive replica selection over awareness attribute based routing

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor changes

Signed-off-by: jyothsna <donapv@amazon.com>

* Address review comments

Signed-off-by: jyothsna <donapv@amazon.com>

* Fix checkstyle

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor fix

Signed-off-by: jyothsna <donapv@amazon.com>
2022-01-21 11:11:49 -06:00
Sarat Vemulapalli 781156471a
Removing lingering transportclient (#1955)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-01-21 01:00:50 -08:00
Suraj Singh 27e05cc1dc
[BUG] Wait for outstanding requests to complete in LastSuccessfulSett… (#1939)
* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-20 17:26:53 -06:00
Ankit Jain 6eb8f6f307
Prioritize primary shard movement during shard allocation (#1445)
When some node or set of nodes is excluded (based on some cluster setting) 
BalancedShardsAllocator iterates over them in breadth first order picking 1 shard from 
each node and repeating the process until all shards are balanced. Since shards from 
each node are picked randomly it's possible the p and r of shard1 is relocated first 
leaving behind both p and r of shard2. If the excluded nodes were to go down the 
cluster becomes red. 

This commit introduces a new setting  "cluster.routing.allocation.move.primary_first" 
that prioritizes the p of both shard1 and shard2 first so the cluster does not become 
red if the excluded nodes were to go down before relocating other shards. Note that
with this setting enabled performance of this change is a direct function of number 
of indices, shards, replicas, and nodes. The larger the indices, replicas, and 
distribution scale, the slower the allocation becomes. This should be used with care.

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2022-01-20 17:02:54 -06:00
Tianli Feng 8b8d04173c
Update protobuf-java to 3.19.3 (#1945)
* Update protobuf-java to 3.19.3

Signed-off-by: Tianli Feng <ftl94@live.com>

* Exclude some API usage violations in the package com.google.protobuf for thirdPartyAudit task to pass

Signed-off-by: Tianli Feng <ftl94@live.com>
2022-01-20 11:05:28 -08:00
Vacha e0216dad0f
Using custom branch name for auto backporting PRs (#1862)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-19 16:23:43 -08:00
Andriy Redko 32f2189686
Update Netty to 4.1.73.Final (#1936)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-18 16:30:18 -08:00
Poojita Raj 4a8c51f15f
Add release notes for 1.2.4 (#1934)
Signed-off-by: Poojita Raj <poojiraj@amazon.com>
2022-01-18 16:23:11 -08:00
Nick Knize 7058256e45
Enforce soft deletes (#1903)
This commit enforces soft deletes as required in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-18 12:54:42 -06:00
Suraj Singh 7691f075cb
[Bug] Wait for outstanding requests to complete (#1925)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-18 10:49:47 -05:00
dependabot[bot] d3a4c47b69
Bump com.gradle.enterprise from 3.8 to 3.8.1 (#1920)
Bumps com.gradle.enterprise from 3.8 to 3.8.1.

---
updated-dependencies:
- dependency-name: com.gradle.enterprise
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-18 10:46:18 -05:00
Nick Knize 81d998dd7a
[BWC] Ensure 2.x compatibility with Legacy 7.10.x (#1902)
* [BWC] Ensure 2.x compatibility with Legacy 7.10.x

This commit fixes TransportHandshaker to send a spoofed Legacy 7.10.2 mincompat
version to ensure OpenSearch 2.x nodes can join a Legacy 7.10.x cluster for
rolling upgrade support. Without this change 7.10.x and OpenSearch 2.x mixed
cluster bwc tests would fail.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix AbstractSimpleTransportTestCase failures

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* change v3 to local static

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* make V_3_0_0 package private and reuse in test

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-17 12:26:28 -05:00
Andriy Redko dd8ad29770
Fixing org.opensearch.common.network.InetAddressesTests.testForStringIPv6WithScopeIdInput (#1913)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 15:23:33 -06:00
Matt Weber 524448fc41
Expand SearchPlugin javadocs. (#1909)
Add and clarify some search plugin point documentation.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 13:06:32 -06:00
Andriy Redko f059738aac
[BUG] Serialization bugs can cause node drops (#1885)
This commit restructures InboundHandler to ensure all data 
is consumed over the wire.

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 13:02:34 -06:00
Matt Weber e7d44c20e9
Make SortBuilders pluggable (#1856)
Add the ability for plugin authors to add custom sort builders.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 11:06:13 -06:00
Andriy Redko 6dcfe8cdcc
Fix o.o.transport.netty4.OpenSearchLoggingHandlerIT stack overflow test failure (#1900)
Attempt to fix o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow by 
hardening test expectation patterns in regex patterns

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 10:52:57 -06:00
Tyler 0f31d62cd9
File name correction to follow existing convention (#1874)
Updated the name of the file to follow existing naming conventions. I discovered this due to the 1.2.3 hyperlink on "/docs/1.2/version-history/" being broken.

Signed-off-by: Tyler McVaney <tylermcvaney@gmail.com>
2022-01-13 10:48:02 -05:00
Sarat Vemulapalli 292efe2040
Added help to build distributions in docs (#1898)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-01-13 10:01:54 -05:00
Owais Kazi cfc9ec292d
Linked the formatting setting file (#1860)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-01-13 10:00:32 -05:00
Nick Knize 0887ac6d92
[Remove] Old Translog Checkpoint Format (#1884)
This commit removes support for old translog checkpoint formats prior to 7.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-11 12:58:35 -06:00
dependabot[bot] c457272e6b
Bump com.gradle.enterprise from 3.5 to 3.8 (#1880)
Bumps com.gradle.enterprise from 3.5 to 3.8.

---
updated-dependencies:
- dependency-name: com.gradle.enterprise
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-11 10:15:43 -05:00
Rabi Panda 8fbd8cd998
Fix verifyVersions gradle task and cleanup bwcVersions (#1878)
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:51 -05:00
Rabi Panda 40442aae7f
Remove unwanted unreleased versions (#1877)
Only last major (2.0.0), last minor(1.3.0) and (1.2.4) can be unreleased

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:24 -05:00
Vacha 412bf82a4a
Updating dependabot open pr limits (#1875)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-11 09:18:53 -05:00
Rabi Panda 28f688fbc5
Use Version.compareMajor instead of using equals operator (#1876)
This test fails due to equality check of majors between OpenSearch Version 1.x and LegacyESVersion 7.x. They should be equivalent but in this case the check fails and LegacyVersion.v6.8.x is chosen which was removed in 52508d5.

This change fixes the comparison logic by using `Version.compareMajor` method instead.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 17:59:08 -08:00
Andriy Redko c192ecdfec
Attempt to fix :test:fixtures:s3-fixture:composeUp fails due to HTTP connection issue (#1866)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-10 17:34:52 -06:00
Andriy Redko 4dbf1d268c
Concurrent Searching (Experimental): modify profiling implementation to support concurrent data collection (#1673)
* Concurrent Searching (Experimental): modify profiling implementation to support concurrent data collection

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-10 17:11:48 -06:00
Rabi Panda ec24ee56e8
Remove deprecated search.remote settings (#1870)
This commit removes the deprecated search.remote.* settings.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 11:45:56 -06:00
Rabi Panda 52508d57f5
Remove LegacyESVersion.V_6_8_x constants (#1869)
This commit removes all usages of the `LegacyESVersion.V_6_8_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-10 11:43:15 -06:00
Andriy Redko c71ff87303
Fixing build failures after Flavor Serialization backport (#1867)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-07 16:43:56 -08:00
sharp-pixel 1bb8864178
Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions. (#1668)
* Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>

* Fix dangling HTML tags in javadoc to solve build issues with JDK 17.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>
2022-01-06 12:39:13 -05:00
Tianli Feng db23f72a2a
Update FIPS API libraries of Bouncy Castle (#1853)
* Update bc-fips to 1.0.2.1

Signed-off-by: Tianli Feng <ftl94@live.com>

* Update bcpg-fips to 1.0.5.1

Signed-off-by: Tianli Feng <ftl94@live.com>

* Update bctls-fips to 1.0.12.2

Signed-off-by: Tianli Feng <ftl94@live.com>

* Use the unified bouncycastle version for bcpkix-jdk15on in HDFS testing fixture

Signed-off-by: Tianli Feng <ftl94@live.com>
2022-01-05 16:59:55 -08:00
Vacha 6e26679274
Fixing typo in TESTING.md (#1849)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-05 08:17:26 -05:00
Daniel Doubrovkine (dB.) 7a38109f4a
Auto-increment next development iteration. (#1816)
* Auto-increment next development iteration.

Signed-off-by: dblock <dblock@amazon.com>

* Make bwc increments on X.Y and main branches.

Signed-off-by: dblock <dblock@amazon.com>
2022-01-05 08:15:12 -05:00
Vacha b99f10aaa1
Fixing auto backport workflow (#1845)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-04 15:22:54 -08:00
CEHENKLE 2fb83208e4
Catching Maintainers up for Q4 2021 new additions/removals (#1841)
Signed-off-by: CEHENKLE <henkle@amazon.com>
2022-01-04 08:33:50 -05:00
Ashish Agrawal 0b2eba568d
Update junit to 4.13.1 (#1837)
* Update junit to 4.13.1

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

* update junit to 4.13.2

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

* update SHA1 file

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>
2022-01-04 08:27:19 -05:00
Zelin Hao e67ba91a92
Change whitesource integration to scan on 1.x branch (#1786)
* Change whitesource integration to scan on 1.x branch

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Add parameter to ignore /qa/ folder

Signed-off-by: Zelin Hao <zelinhao@amazon.com>
2022-01-03 17:49:09 -08:00
Andriy Redko 385b268bc0
Update Mockito to 4.2.x (#1830)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-03 12:00:45 -05:00
Sarat Vemulapalli db91d2efe9
Upgrading bouncycastle to 1.70 (#1832) 2022-01-03 07:35:38 -05:00
Sarat Vemulapalli 7a97018a92
Updatting Netty to 4.1.72.Final (#1831) 2022-01-03 07:30:52 -05:00
Rabi Panda 06cbc47136
Remove LegacyESVersion.V_6_7_x constants (#1807)
This commit removes all usages of the `LegacyESVersion.V_6_7_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-02 14:09:55 -08:00
Rabi Panda 5ccb22fb4b
Remove LegacyESVersion.V_6_6_x constants (#1804)
This commit removes all usages of the `LegacyESVersion.V_6_6_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-29 22:43:53 -08:00
Shinsuke Sugaya d843ddd9c0
replace with opensearch-http-channel and opensearch-http-server-channel (#1799)
Signed-off-by: Shinsuke Sugaya <shinsuke@apache.org>
2021-12-29 11:32:49 -06:00
Andriy Redko abe4b97199
Execution failed for task ':test:fixtures:azure/s3/hdfs/gcs-fixture:composeDown' (#1824)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-29 11:26:14 -05:00
Tianli Feng c05e0d90bd
[TEST] Fix FsHealthServiceTest by increasing the timeout period before checking the FS health after restoring the FS status (#1813)
Signed-off-by: Tianli Feng <ftl94@live.com>
2021-12-28 20:50:59 -08:00
Andriy Redko 65804d25a6
Update to log4j 2.17.1 (#1820)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-28 17:06:42 -05:00
Andriy Redko 65813b8320
RestIntegTestTask fails because of missed log4j-core dependency (#1815)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-28 14:51:25 -05:00
Rabi Panda 01e1b94e47
Remove LegacyESVersion.V_6_5_x constants (#1794)
* Remove v6.5.x constants

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

* Address code review comments.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

* Add back version assumption for testRemoteClusterSettingsUpgraded

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

* Remove unused import after rebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-28 11:52:51 -06:00