Commit Graph

55365 Commits

Author SHA1 Message Date
Andrey Pleskach b9ff91d591
Add proxy settings for GCS repository (#2096)
Added proxy settings for GCS repository.
Security settings:
- gcs.client.*.proxy.username - Proxy user name
- gcs.client.*.proxy.password - Proxy user password

Common settings:
- gcs.client.*.proxy.type - java Proxy.Type names: HTTP, SOCKS. default is DIRECT
- gcs.client.*.proxy.host - Proxy host name
- gcs.client.*.proxy.port - Proxy port

Signed-off-by: Andrey Pleskach <ples@aiven.io>
2022-02-17 11:49:33 -08:00
Peng Huo 9ea25c4f2e
Adjust main version after backport to 1.x (#2147)
Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-02-17 12:17:10 -06:00
dependabot[bot] 9689a27b63
Bump com.diffplug.spotless from 5.6.1 to 6.2.0 (#1919)
* Bump com.diffplug.spotless from 5.6.1 to 6.2.0

Bumps com.diffplug.spotless from 5.6.1 to 6.2.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-16 22:08:40 -08:00
Vacha Shah d47725d9c3
Dependabot fix and pr actions (#1950)
* Updating the dependabot paths for nested directories

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Adding a workflow for running actions on the dependabot generated PR

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Changing limits and using Github App token

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-02-16 12:01:25 -05:00
Nick Knize c0c7d7aedb
[Remove] Type parameter from TermVectors API (#2104)
Remove document type support from the TermVectors API.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-15 14:49:08 -06:00
Andrey Pleskach 62361ceafc
Add proxy username and password settings for Azure repository (#2098)
Added username/password proxy settings for Azure repository.
Security settings:
- azure.client.*.proxy.username - Proxy user name
- azure.client.*.proxy.password - Proxy user password

Signed-off-by: Andrey Pleskach <ples@aiven.io>
2022-02-15 10:40:47 -08:00
Andriy Redko 6b6f03368f
Mapping update for “date_range” field type is not idempotent (#2094)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-14 16:56:02 -06:00
Andriy Redko a942b275ef
Fixing org.opensearch.monitor.os.OsProbeTests::testLogWarnCpuMessageOnlyOnes when CGroups are not available (#2101)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-14 12:10:16 -05:00
Andrey Pleskach 27ed6fc82c
Fix integration tests failure (#2067)
Fixed integration tests failure on Linux with Kernel 5.16.x

Signed-off-by: Andrey Pleskach <ples@aiven.io>
2022-02-11 09:40:04 -05:00
Suraj Singh 892801a074
Remove type mapping from document index API (#2026)
* Remove doc type specific indexing APIs and relevant changes

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Remove type param from yml and integ test files

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-09 13:34:10 -06:00
Ankit Jain f0984eb409
Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation (#2074)
Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2022-02-09 00:55:30 -08:00
Vacha bb4d06cf1f
Using Github App to trigger CI for auto-backport (#2071)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-02-08 15:06:32 -08:00
Kunal Kotwani dd9e7cfa31
Add JetBrains Gateway setup details (#1944)
* Add JetBrains Gateway setup details

Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>

* Moving JetBrains Gateway setup under IntelliJ remote development

Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
2022-02-08 16:18:01 -05:00
Nick Knize 6428d1a2a6
[Refactor] MapperService to QueryShardContext in valueFetcher (#2027)
* [Refactor] MapperService to QueryShardContext in valueFetcher

QueryShardContext encapsulates MapperService along with other mechanisms useful
for analyzers, highlighters, and readers. This commit refactors the valueFetcher
to use the QueryShardContext as a parameter instead of direct access to the
MapperService. This is an API change. Behavior of the method is unaffected.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* change unnecessary Optional usage

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-08 08:38:20 -05:00
Ankit Jain 343b82fe24
Stabilizing org.opensearch.cluster.routing.MovePrimaryFirstTests.test… (#2048)
* Stabilizing org.opensearch.cluster.routing.MovePrimaryFirstTests.testClusterGreenAfterPartialRelocation

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>

* Removing unused import

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>

* Making code more readable

Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
2022-02-07 19:10:07 -08:00
Matt Weber b9420d8f70
Add regexp interval source (#1917)
* Add regexp interval source

Add a regexp interval source provider so people can use regular
expressions inside of intervals queries.

Signed-off-by: Matt Weber <matt@mattweber.org>

* Fixes

- register regexp interval in SearchModule
- use fully-qualified name for lucene RegExp
- get rid of unnecessary variable

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-02-07 17:18:43 -06:00
Vacha 9c9e218ae6
Adding workflow to auto delete backport merged branches from backport workflow (#2050)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-02-07 13:12:29 -08:00
Nick Knize 8288c9010c
[Remove] CircuitBreaker Accounting (#2056)
RAM Accounting of segments is removed so remove the associated Accounting
circuit breaker.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-04 19:46:27 -06:00
Nicholas Walter Knize ee9e5ba579
Add Version.V_1_2_5 constant
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-04 18:21:51 -06:00
Nicholas Walter Knize 9c4caec4b7
add 1.2.5 to bwcVersions
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-04 15:10:18 -06:00
Nick Knize 25955782b9
[Refactor] Lucene DataInput and DataOutput to StreamInput and StreamOutput (#2035)
Lucene 9 changes from BigEndian to LittleEndian in DataInput and DataOutput.
The use of Lucene's Data IO classes are refactored in this commit to use
StreamInput which maintains the same method calls while preserving the byte
order of the data.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-04 14:23:18 -06:00
Nick Knize fc0d3a368d
[Remove] Segment memory estimation and tracking (#2029)
Lucene 9 removed CodecReader#ramBytesUsed and all file formats that no longer
consume large amounts of memory. As a result RAM estimation for segments is no
longer possible and is removed by this commit. backwards compatibility is retained 
for the tranport layer.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-04 13:47:48 -06:00
Andriy Redko 1e5d98329e
Another attempt to fix o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow (#2051)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-04 11:23:36 -05:00
Martin Gaievski 309397520c
Adding support for JDK17 and removing JDK8 (#2025)
* Adding support for JDK17 and removing JDK8

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Merge overlaping PR, bumping min java version to 11

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Removing references to JDK8 from dev guide

Signed-off-by: Martin Gaievski <gaievski@amazon.com>
2022-02-02 20:59:10 -05:00
Lukáš Vlček 270c59f523
Fix AssertionError message (#2044)
Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
2022-02-02 16:58:25 -06:00
Andriy Redko c8ac037389
[FEATURE] Add OPENSEARCH_JAVA_HOME env to override JAVA_HOME (#2001)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-02 13:55:22 -05:00
Leonidas Spyropoulos 6da253b8ff
build: introduce support for reproducible builds (#1995)
Reproducible builds is an initiative to create an independently-verifiable path from source to binary code [1]. This can be done by:
- Make all archive tasks in gradle reproducible by ignoring timestamp on files [2]
- Preserve the order in side the archives [2]
- Ensure GlobalBuildInfoPlugin.java use [SOURCE_DATE_EPOCH] when available

[SOURCE_DATE_EPOCH]: https://reproducible-builds.org/docs/source-date-epoch/
[1]: https://reproducible-builds.org/
[2]: https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2022-02-02 09:13:54 -06:00
Sarat Vemulapalli 1f9517c4ca
Upgrading Shadow plugin to 7.1.2 (#2033)
Shadow plugin is used for publishing jars
and this upgrades Log4J dependency for build.

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-02-02 08:51:23 -06:00
Peng Huo 2fc8acd409
Fix composite aggregation failed test cases introduce by missing_order parameter (#1942) (#2005)
Fix composite aggregation failed test cases introduce by missing_order parameter by using MissingOrder to decide null values's order in InternalBucket.

Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-02-01 14:19:10 -06:00
Andriy Redko 6c2f01a045
[BUG] Docker distribution builds are failing. Switching to http://vault.centos.org (#2024)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-01 12:42:48 -06:00
Lukáš Vlček a597d4daac
Fixing invalid Java code example in JavaDoc (#2008)
Including several other minor typos in JavaDoc.

Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
2022-01-30 16:31:42 -08:00
Andriy Redko 00416dc428
Update bundled JDK distribution to 17.0.2+8 (#2007)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-28 09:52:56 -08:00
Nick Knize 5569ce0025
[Remove] index.merge.policy.max_merge_at_once_explicit (#1988)
Remove deprecated max_merge_at_once_explicit parameter since it has been
removed in lucene 9 and deprecated in opensearch 1.3

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-27 14:05:13 -06:00
Nick Knize 334f0d9908
[Remove] Setting explicit version on analysis component (#1986)
Lucene 9 removes the ability to define an explicit version on an analysis
component so remove the the version parameter and deprecation warning
prohibiting users from explicitly setting versioned analyzers.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-27 14:04:26 -06:00
Owais Kazi 054595b140
Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation (#1983)
* Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed spotless imports

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-01-27 09:26:32 -08:00
Sergey Nuyanzin e399ac1f27
Add IssueNavigationLink (#1964)
Signed-off-by: Sergey Nuyanzin <snuyanzin@gmail.com>
2022-01-27 09:04:03 -08:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
Nick Knize 0791c88af1
[Remove] Deprecated Synced Flush API (#1761)
Remove the deprecated sync flush API which was replaced by sequence
number and retention lease mechanisms and no longer used in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 22:18:48 -06:00
Nick Knize 0f3b72b7bb
[Deprecate] Setting explicit version on analysis component (#1978)
Lucene 9 removes the ability to define an explicit version on an analysis 
component. The version parameter is deprecated at parse time and a warning is 
issued 
to the user through the deprecation logger.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:46:28 -06:00
Nick Knize a18b4e35b8
[Deprecate] index.merge.policy.max_merge_at_once_explicit (#1981)
max_merge_at_once_explicit is removed in lucene 9 so the index setting is
deprecated for removal in the next major release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:33:52 -06:00
Matt Weber 0bc30b4314
Wildcard max_expansion version check update (#1980)
Change from 2x to 1.3 for wildcard interval max_expansion version check
since #1916 was backported.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-26 18:43:20 -06:00
Peng Huo 7a9314a3bb
Support _first and _last parameter for missing bucket ordering in composite aggregation (#1942)
Adds support for _first and _last parameters for missing bucket ordering in composite aggregation. 
By default, if order is asc, missing_bucket at first, if order is desc, missing_bucket at last. If 
missing_order is "_first" or "_last", regardless order, missing_bucket is at first or last respectively.

Signed-off-by: Peng Huo <penghuo@gmail.com>
2022-01-26 16:36:50 -06:00
Saurabh Singh 447b20c457
Delay the request size calculation until required by the indexing pressure framework (#1592)
Logically delay the request size calculation until it's absolutely required by the 
indexing pressure framework.

Signed-off-by: Saurabh Singh <sisurab@amazon.com>
2022-01-26 15:06:01 -06:00
Dave Lago 6bb560f7f1
Modified TESTING instructions to clarify use of testing classes (#1930)
Improve TESTING instructions to clarify use and purpose of testing classes

Signed-off-by: Dave Lago <davelago@amazon.com>
2022-01-26 14:54:58 -06:00
Andriy Redko 1018b9c8dc
Update Mockito to 4.3.1 (#1973)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-26 14:53:41 -06:00
Rishikesh Pasham 3f92aecddf
Enabling Sort Optimization to make use of Lucene (#1974)
Remove sort optimization. From Lucence 8.10.1, Numeric sort optimization is 
already done in Lucene Comparators. Lucene also supports sort optimization 
with search_after queries.

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 14:36:07 -06:00
Matt Weber 3b8db91abe
Add max_expansions option to wildcard interval (#1916)
Add support for setting the max expansions on a wildcard interval.
The default value is still 128 and the max value is bounded by
`BooleanQuery.getMaxClauseCount()`.

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-01-26 13:06:39 -06:00
Nick Knize 57ac788bb6
[Refactor] InternalEngine to always use soft deletes (#1933)
Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This commit refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 12:28:24 -06:00
ankitkala 56ae7fab63
Add hook to execute logic before Integ test task starts (#1969)
Signed-off-by: Ankit Kala <ankikala@amazon.com>
2022-01-25 14:47:45 +05:30
Vacha 3ce377f5cc
Using pull_request_target in place of pull_request (#1952)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-01-21 12:32:00 -08:00