Commit Graph

55240 Commits

Author SHA1 Message Date
Andriy Redko c8ac037389
[FEATURE] Add OPENSEARCH_JAVA_HOME env to override JAVA_HOME (#2001)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-02 13:55:22 -05:00
Leonidas Spyropoulos 6da253b8ff
build: introduce support for reproducible builds (#1995)
Reproducible builds is an initiative to create an independently-verifiable path from source to binary code [1]. This can be done by:
- Make all archive tasks in gradle reproducible by ignoring timestamp on files [2]
- Preserve the order in side the archives [2]
- Ensure GlobalBuildInfoPlugin.java use [SOURCE_DATE_EPOCH] when available

[SOURCE_DATE_EPOCH]: https://reproducible-builds.org/docs/source-date-epoch/
[1]: https://reproducible-builds.org/
[2]: https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2022-02-02 09:13:54 -06:00
Sarat Vemulapalli 1f9517c4ca
Upgrading Shadow plugin to 7.1.2 (#2033)
Shadow plugin is used for publishing jars
and this upgrades Log4J dependency for build.

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-02-02 08:51:23 -06:00
Peng Huo 2fc8acd409
Fix composite aggregation failed test cases introduce by missing_order parameter (#1942) (#2005)
Fix composite aggregation failed test cases introduce by missing_order parameter by using MissingOrder to decide null values's order in InternalBucket.

Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-02-01 14:19:10 -06:00
Andriy Redko 6c2f01a045
[BUG] Docker distribution builds are failing. Switching to http://vault.centos.org (#2024)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-01 12:42:48 -06:00
Lukáš Vlček a597d4daac
Fixing invalid Java code example in JavaDoc (#2008)
Including several other minor typos in JavaDoc.

Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
2022-01-30 16:31:42 -08:00
Andriy Redko 00416dc428
Update bundled JDK distribution to 17.0.2+8 (#2007)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-28 09:52:56 -08:00
Nick Knize 5569ce0025
[Remove] index.merge.policy.max_merge_at_once_explicit (#1988)
Remove deprecated max_merge_at_once_explicit parameter since it has been
removed in lucene 9 and deprecated in opensearch 1.3

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-27 14:05:13 -06:00
Nick Knize 334f0d9908
[Remove] Setting explicit version on analysis component (#1986)
Lucene 9 removes the ability to define an explicit version on an analysis
component so remove the the version parameter and deprecation warning
prohibiting users from explicitly setting versioned analyzers.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-27 14:04:26 -06:00
Owais Kazi 054595b140
Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation (#1983)
* Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed spotless imports

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-01-27 09:26:32 -08:00
Sergey Nuyanzin e399ac1f27
Add IssueNavigationLink (#1964)
Signed-off-by: Sergey Nuyanzin <snuyanzin@gmail.com>
2022-01-27 09:04:03 -08:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
Nick Knize 0791c88af1
[Remove] Deprecated Synced Flush API (#1761)
Remove the deprecated sync flush API which was replaced by sequence
number and retention lease mechanisms and no longer used in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 22:18:48 -06:00
Nick Knize 0f3b72b7bb
[Deprecate] Setting explicit version on analysis component (#1978)
Lucene 9 removes the ability to define an explicit version on an analysis 
component. The version parameter is deprecated at parse time and a warning is 
issued 
to the user through the deprecation logger.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:46:28 -06:00
Nick Knize a18b4e35b8
[Deprecate] index.merge.policy.max_merge_at_once_explicit (#1981)
max_merge_at_once_explicit is removed in lucene 9 so the index setting is
deprecated for removal in the next major release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:33:52 -06:00
Matt Weber 0bc30b4314
Wildcard max_expansion version check update (#1980)
Change from 2x to 1.3 for wildcard interval max_expansion version check
since #1916 was backported.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-26 18:43:20 -06:00
Peng Huo 7a9314a3bb
Support _first and _last parameter for missing bucket ordering in composite aggregation (#1942)
Adds support for _first and _last parameters for missing bucket ordering in composite aggregation. 
By default, if order is asc, missing_bucket at first, if order is desc, missing_bucket at last. If 
missing_order is "_first" or "_last", regardless order, missing_bucket is at first or last respectively.

Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-01-26 16:36:50 -06:00
Saurabh Singh 447b20c457
Delay the request size calculation until required by the indexing pressure framework (#1592)
Logically delay the request size calculation until it's absolutely required by the 
indexing pressure framework.

Signed-off-by: Saurabh Singh <sisurab@amazon.com>
2022-01-26 15:06:01 -06:00
Dave Lago 6bb560f7f1
Modified TESTING instructions to clarify use of testing classes (#1930)
Improve TESTING instructions to clarify use and purpose of testing classes

Signed-off-by: Dave Lago <davelago@amazon.com>
2022-01-26 14:54:58 -06:00
Andriy Redko 1018b9c8dc
Update Mockito to 4.3.1 (#1973)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-26 14:53:41 -06:00
Rishikesh Pasham 3f92aecddf
Enabling Sort Optimization to make use of Lucene (#1974)
Remove sort optimization. From Lucence 8.10.1, Numeric sort optimization is 
already done in Lucene Comparators. Lucene also supports sort optimization 
with search_after queries.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 14:36:07 -06:00
Matt Weber 3b8db91abe
Add max_expansions option to wildcard interval (#1916)
Add support for setting the max expansions on a wildcard interval.
The default value is still 128 and the max value is bounded by
`BooleanQuery.getMaxClauseCount()`.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-26 13:06:39 -06:00
Nick Knize 57ac788bb6
[Refactor] InternalEngine to always use soft deletes (#1933)
Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This commit refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 12:28:24 -06:00
ankitkala 56ae7fab63
Add hook to execute logic before Integ test task starts (#1969)
Signed-off-by: Ankit Kala <ankikala@amazon.com>
2022-01-25 14:47:45 +05:30
Vacha 3ce377f5cc
Using pull_request_target in place of pull_request (#1952)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-21 12:32:00 -08:00
Venkata Jyothsna Donapati 02c6a7f156
Prefer adaptive replica selection over awareness attribute based routing (#1107)
* Prefer adaptive replica selection over awareness attribute based routing

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor changes

Signed-off-by: jyothsna <donapv@amazon.com>

* Address review comments

Signed-off-by: jyothsna <donapv@amazon.com>

* Fix checkstyle

Signed-off-by: jyothsna <donapv@amazon.com>

* Minor fix

Signed-off-by: jyothsna <donapv@amazon.com>
2022-01-21 11:11:49 -06:00
Sarat Vemulapalli 781156471a
Removing lingering transportclient (#1955)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-01-21 01:00:50 -08:00
Suraj Singh 27e05cc1dc
[BUG] Wait for outstanding requests to complete in LastSuccessfulSett… (#1939)
* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-20 17:26:53 -06:00
Ankit Jain 6eb8f6f307
Prioritize primary shard movement during shard allocation (#1445)
When some node or set of nodes is excluded (based on some cluster setting) 
BalancedShardsAllocator iterates over them in breadth first order picking 1 shard from 
each node and repeating the process until all shards are balanced. Since shards from 
each node are picked randomly it's possible the p and r of shard1 is relocated first 
leaving behind both p and r of shard2. If the excluded nodes were to go down the 
cluster becomes red. 

This commit introduces a new setting  "cluster.routing.allocation.move.primary_first" 
that prioritizes the p of both shard1 and shard2 first so the cluster does not become 
red if the excluded nodes were to go down before relocating other shards. Note that
with this setting enabled performance of this change is a direct function of number 
of indices, shards, replicas, and nodes. The larger the indices, replicas, and 
distribution scale, the slower the allocation becomes. This should be used with care.

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2022-01-20 17:02:54 -06:00
Tianli Feng 8b8d04173c
Update protobuf-java to 3.19.3 (#1945)
* Update protobuf-java to 3.19.3

Signed-off-by: Tianli Feng <ftl94@live.com>

* Exclude some API usage violations in the package com.google.protobuf for thirdPartyAudit task to pass

Signed-off-by: Tianli Feng <ftl94@live.com>
2022-01-20 11:05:28 -08:00
Vacha e0216dad0f
Using custom branch name for auto backporting PRs (#1862)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-19 16:23:43 -08:00
Andriy Redko 32f2189686
Update Netty to 4.1.73.Final (#1936)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-18 16:30:18 -08:00
Poojita Raj 4a8c51f15f
Add release notes for 1.2.4 (#1934)
Signed-off-by: Poojita Raj <poojiraj@amazon.com>
2022-01-18 16:23:11 -08:00
Nick Knize 7058256e45
Enforce soft deletes (#1903)
This commit enforces soft deletes as required in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-18 12:54:42 -06:00
Suraj Singh 7691f075cb
[Bug] Wait for outstanding requests to complete (#1925)
Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-01-18 10:49:47 -05:00
dependabot[bot] d3a4c47b69
Bump com.gradle.enterprise from 3.8 to 3.8.1 (#1920)
Bumps com.gradle.enterprise from 3.8 to 3.8.1.

---
updated-dependencies:
- dependency-name: com.gradle.enterprise
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-18 10:46:18 -05:00
Nick Knize 81d998dd7a
[BWC] Ensure 2.x compatibility with Legacy 7.10.x (#1902)
* [BWC] Ensure 2.x compatibility with Legacy 7.10.x

This commit fixes TransportHandshaker to send a spoofed Legacy 7.10.2 mincompat
version to ensure OpenSearch 2.x nodes can join a Legacy 7.10.x cluster for
rolling upgrade support. Without this change 7.10.x and OpenSearch 2.x mixed
cluster bwc tests would fail.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix AbstractSimpleTransportTestCase failures

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* change v3 to local static

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* make V_3_0_0 package private and reuse in test

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-17 12:26:28 -05:00
Andriy Redko dd8ad29770
Fixing org.opensearch.common.network.InetAddressesTests.testForStringIPv6WithScopeIdInput (#1913)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 15:23:33 -06:00
Matt Weber 524448fc41
Expand SearchPlugin javadocs. (#1909)
Add and clarify some search plugin point documentation.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 13:06:32 -06:00
Andriy Redko f059738aac
[BUG] Serialization bugs can cause node drops (#1885)
This commit restructures InboundHandler to ensure all data 
is consumed over the wire.

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 13:02:34 -06:00
Matt Weber e7d44c20e9
Make SortBuilders pluggable (#1856)
Add the ability for plugin authors to add custom sort builders.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-14 11:06:13 -06:00
Andriy Redko 6dcfe8cdcc
Fix o.o.transport.netty4.OpenSearchLoggingHandlerIT stack overflow test failure (#1900)
Attempt to fix o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow by 
hardening test expectation patterns in regex patterns

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 10:52:57 -06:00
Tyler 0f31d62cd9
File name correction to follow existing convention (#1874)
Updated the name of the file to follow existing naming conventions. I discovered this due to the 1.2.3 hyperlink on "/docs/1.2/version-history/" being broken.

Signed-off-by: Tyler McVaney <tylermcvaney@gmail.com>
2022-01-13 10:48:02 -05:00
Sarat Vemulapalli 292efe2040
Added help to build distributions in docs (#1898)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-01-13 10:01:54 -05:00
Owais Kazi cfc9ec292d
Linked the formatting setting file (#1860)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-01-13 10:00:32 -05:00
Nick Knize 0887ac6d92
[Remove] Old Translog Checkpoint Format (#1884)
This commit removes support for old translog checkpoint formats prior to 7.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-11 12:58:35 -06:00
dependabot[bot] c457272e6b
Bump com.gradle.enterprise from 3.5 to 3.8 (#1880)
Bumps com.gradle.enterprise from 3.5 to 3.8.

---
updated-dependencies:
- dependency-name: com.gradle.enterprise
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-11 10:15:43 -05:00
Rabi Panda 8fbd8cd998
Fix verifyVersions gradle task and cleanup bwcVersions (#1878)
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:51 -05:00
Rabi Panda 40442aae7f
Remove unwanted unreleased versions (#1877)
Only last major (2.0.0), last minor(1.3.0) and (1.2.4) can be unreleased

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:24 -05:00
Vacha 412bf82a4a
Updating dependabot open pr limits (#1875)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-11 09:18:53 -05:00