Commit Graph

5865 Commits

Author SHA1 Message Date
Nick Knize 2e3f410957
[Remove] Types from GET/MGET (#2168)
Removes type support from the get and mget transport and rest actions along
with removal from ShardGetService.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-21 11:37:27 -06:00
dependabot[bot] a9324786ee
Bump compiler from 0.9.6 to 0.9.10 in /modules/lang-mustache (#2126)
* Bump compiler from 0.9.6 to 0.9.10 in /modules/lang-mustache

Bumps [compiler](https://github.com/spullara/mustache.java) from 0.9.6 to 0.9.10.
- [Release notes](https://github.com/spullara/mustache.java/releases)
- [Commits](https://github.com/spullara/mustache.java/compare/mustache.java-0.9.6...0.9.10)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
2022-02-18 15:27:07 -08:00
QUANG MAU BACH 0e95bb9dff
Add Factory to enable Lucene ConcatenateGraphFilter (#1278) (#2152)
Lucene has a ConcatenateGraphFilter that can concatenate tokens from a TokenStream
to create a single token (or several tokens that have the same position if
input TokenStream is a graph).

The change is to enable that ConcatenateGraphFilter by adding a Factory.

Signed-off-by: Mau Bach Quang <quangmaubach@gmail.com>
2022-02-18 10:05:44 -08:00
Nick Knize b75d3f1123
[Remove] Default Mapping (#2151)
Default mappings were deprecated in Legacy version 6x and cannot be added to
indexes created in legacy 7+ or any version of OpenSearch. All support for
default mappings are removed for OpenSearch 2+.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-17 22:21:20 -06:00
Nick Knize 7fe642fda5
[Remove] Type from Search Internals (#2109)
With types deprecation the type support is removed from internal search API
(SearchRequest and QueryShardContext).

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-17 16:17:31 -06:00
dependabot[bot] 9689a27b63
Bump com.diffplug.spotless from 5.6.1 to 6.2.0 (#1919)
* Bump com.diffplug.spotless from 5.6.1 to 6.2.0

Bumps com.diffplug.spotless from 5.6.1 to 6.2.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-16 22:08:40 -08:00
Andriy Redko 6b6f03368f
Mapping update for “date_range” field type is not idempotent (#2094)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-14 16:56:02 -06:00
Suraj Singh 892801a074
Remove type mapping from document index API (#2026)
* Remove doc type specific indexing APIs and relevant changes

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Remove type param from yml and integ test files

Signed-off-by: Suraj Singh <surajrider@gmail.com>
2022-02-09 13:34:10 -06:00
Nick Knize 6428d1a2a6
[Refactor] MapperService to QueryShardContext in valueFetcher (#2027)
* [Refactor] MapperService to QueryShardContext in valueFetcher

QueryShardContext encapsulates MapperService along with other mechanisms useful
for analyzers, highlighters, and readers. This commit refactors the valueFetcher
to use the QueryShardContext as a parameter instead of direct access to the
MapperService. This is an API change. Behavior of the method is unaffected.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* change unnecessary Optional usage

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-08 08:38:20 -05:00
Andriy Redko 1e5d98329e
Another attempt to fix o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow (#2051)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-02-04 11:23:36 -05:00
Nick Knize 334f0d9908
[Remove] Setting explicit version on analysis component (#1986)
Lucene 9 removes the ability to define an explicit version on an analysis
component so remove the the version parameter and deprecation warning
prohibiting users from explicitly setting versioned analyzers.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-27 14:04:26 -06:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
Nick Knize 0f3b72b7bb
[Deprecate] Setting explicit version on analysis component (#1978)
Lucene 9 removes the ability to define an explicit version on an analysis 
component. The version parameter is deprecated at parse time and a warning is 
issued 
to the user through the deprecation logger.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:46:28 -06:00
Andriy Redko 32f2189686
Update Netty to 4.1.73.Final (#1936)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-18 16:30:18 -08:00
Andriy Redko 6dcfe8cdcc
Fix o.o.transport.netty4.OpenSearchLoggingHandlerIT stack overflow test failure (#1900)
Attempt to fix o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow by 
hardening test expectation patterns in regex patterns

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-14 10:52:57 -06:00
sharp-pixel 1bb8864178
Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions. (#1668)
* Add darwin-arm64-tar and no-jdk-darwin-arm64-tar archive distributions.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>

* Fix dangling HTML tags in javadoc to solve build issues with JDK 17.

Signed-off-by: Cédric Pelvet <cedric.pelvet@gmail.com>
2022-01-06 12:39:13 -05:00
Andriy Redko 385b268bc0
Update Mockito to 4.2.x (#1830)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-03 12:00:45 -05:00
Sarat Vemulapalli 7a97018a92
Updatting Netty to 4.1.72.Final (#1831) 2022-01-03 07:30:52 -05:00
Rabi Panda 06cbc47136
Remove LegacyESVersion.V_6_7_x constants (#1807)
This commit removes all usages of the `LegacyESVersion.V_6_7_x` constants from the codebase.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-02 14:09:55 -08:00
Shinsuke Sugaya d843ddd9c0
replace with opensearch-http-channel and opensearch-http-server-channel (#1799)
Signed-off-by: Shinsuke Sugaya <shinsuke@apache.org>
2021-12-29 11:32:49 -06:00
Rabi Panda 7c73ed8d4c
Remove deprecated transport client (#1781)
This commit removes all usage of transport client in OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-28 11:28:51 -06:00
Rabi Panda 24d0c9b818
Remove transport client from tests. (#1809)
Transport client has been deprecated. In this commit, we are removing all its usages from the tests.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-27 17:43:18 -06:00
Rabi Panda 8b076b81ee
Remove LegacyVersion.v6.4.x constants (#1787)
This commit removes the `LegacyVersion.v6.4.x` constants and their usages, since we no longer need to support those in OpenSearch 2.0 onwards.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-23 16:54:37 -08:00
Andrew Ross 96d55966a2
Use try-with-resources with MockLogAppender (#1595)
I previously added a helper that started a MockLogAppender to ensure it
was never added to a Logger before it was started. I subsequently found
the opposite case in RolloverIT.java where the appender was stopped
before it was closed, therefore creating a race where a concurrently
running test in the same JVM could cause a logging failure. This seems
like a really easy mistake to make when writing a test or introduce when
refactoring a test. I've made a change to use try-with-resources to
ensure that proper setup and teardown is done. This should make it much
harder to introduce this particular test bug in the future.
Unfortunately, it did involve touching a lot of files. The changes here
are purely structural to leverage try-with-resources; no testing logic
has been changed.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-12-21 15:00:55 -06:00
Nick Knize 5550f8d7e2
[Remove] Analyzer Deprecations (#1741)
This commit removes deprecated analyzer instantiation that is no longer
permitted in OpenSearch 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-16 12:51:49 -06:00
Megha Sai Kavikondala 22bfadf324
Adding a cancelled field to tell if a cancellable task is cancelled (#1732)
* Changes made by adding the Task indicator and the IllegalArgs.

Signed-off-by: meghasaik <kavmegha@amazon.com>

* Changes made by changing the String message for uncancellable task.

Signed-off-by: meghasaik <kavmegha@amazon.com>
2021-12-16 09:49:49 -05:00
Vacha e66ea2c4f3
Avoid logging duplicate deprecation warnings multiple times (#1660)
* Avoid logging duplicate deprecation warnings multiple times

Signed-off-by: Vacha <vachshah@amazon.com>

* Fixes test failures

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding deprecation logger tests

Signed-off-by: Vacha <vachshah@amazon.com>

* Using ConcurrentHashMap keySet

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-12-15 15:26:44 -08:00
Nick Knize 538b40a0dc
Remove LegacyESVersion.V_6_2_x constants (#1686)
This commit removes LegacyESVersion.V_6_2_x constants including all
pre-release versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-09 16:08:00 -06:00
Nick Knize 821417b9ab
Remove LegacyESVersion.V_6_1_x constants (#1681)
This commit removes LegacyESVersion.V_6_1_x constants including all
pre-release versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-08 23:11:36 -06:00
Nick Knize b74d71fb74
Remove 6.0.* version constants (#1658)
This PR removes LegacyESVersion.V_6_0_* constants including all pre-release
versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-08 13:37:21 -06:00
Andriy Redko 8bf6fa6b4f
Support Gradle 7 (#1609)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-29 14:34:06 -05:00
Rishikesh Pasham 0c21689202
Renaming slave to replica in filebeat-6.0.template.json file. (#1569)
Signed-off-by: Rishikesh Pasham <rishireddy1159@gmail.com>
2021-11-22 17:59:56 -05:00
Andrew Ross d467283475
Start MockLogAppender before adding to static context (#1587)
I observed a test failure with the message
'Attempted to append to non-started appender mock' from an assertion in
`OpenSearchTestCase::after`. I believe this indicates that a
MockLogAppender (which is named "mock") was added as an appender to the
static logging context and some other test in the same JVM happened to
cause a logging statement to hit that appender and cause an error, which
then caused an unrelated test to fail (because they share static state
with the logger). Almost all usages of MockLogAppender start it
immediately after creation. I found a few that did not and fixed those.
I also made a static helper in MockLogAppender to start it upon
creation.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-11-19 13:37:46 -05:00
Nick Knize 53334b2ce4
Upgrade to Lucene 8.10.1 (#1440)
This commit upgrades to the latest release of lucene 8.10

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-10-28 10:06:53 -05:00
Owais Kazi 37ac3788a3
Run spotless and exclude checkstyle on modules module (#1442)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-27 13:24:43 -07:00
Xue Zhou 574e42c31b
Remove old ES libraries used in reindex due to CVEs (#1359)
This commit removes old ES libraries version 090 and 176 due to CVE

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-10-20 11:51:45 -05:00
Vacha c7f5c90a5f
Upgrading netty version to 4.1.69.Final (#1363)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-10-18 17:30:19 -04:00
Andriy Redko e9635d6bfe
Replace securemock with mock-maker (test support), update Mockito to 3.12.4 (#1332)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-10 14:18:54 -04:00
Andriy Redko cdbc84f09d
Update Jackson to 2.12.5 (#1247)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-21 18:33:20 -04:00
Andriy Redko b6c8bdf872
Drop mocksocket in favour of custom security manager checks (tests only) (#1205)
* Drop mocksocket in favour of custom security manager checks (tests only)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Slightly relaxed host checks to allow all local addresses

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-16 17:21:47 -04:00
Andriy Redko 0b773c2cec
Fix org.opensearch.index.reindex.ReindexRestClientSslTests#testClientSucceedsWithCertificateAuthorities - javax.net.ssl.SSLPeerUnverifiedException (#1212)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-08 09:08:05 -04:00
Nick Knize 5ae00456a0
Upgrade to Lucene 8.9 (#1080)
This commit upgrades to the official lucene 8.9 release

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-20 11:28:06 -05:00
Yevhen Tienkaiev f46f6950ec
Rank feature - unknown field linear (#983)
Signed-off-by: Yevhen Tienkaiev <hronom@gmail.com>
2021-07-29 14:28:29 -04:00
Sooraj Sinha a428e6f043
Pass interceptor to super constructor (#876)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-07-05 11:13:36 +05:30
Tianli Feng 110cef7882
Fix resource leak issues suggested by Amazon CodeGuru (#816)
* Address a kind of issue suggested by Amazon CodeGuru Reviewer:

* Add try-with-resources block to automatically close the resources after using to avoid resource leak, in `SymbolicLinkPreservingTarIT`, `LicenseAnalyzer`, `SymbolicLinkPreservingUntarTransform`, `ConcurrentSeqNoVersioningIT` in `VersionProperties`, `GeoFilterIT`, `XContentHelper`,  `Json` and `IndexShard` class

* Add try-finally block to close the resources after using to avoid resource leak, in `ServerChannelContext` class.

* Add try-catch block to close the resources when exception occurs in `FsBlobContainer` class (when XContentFactory.xContentType throws an exception).

* Close resources when assertion error occurs, in `ServerChannelContext` class.
2021-06-15 09:31:36 -07:00
Nick Knize d821b15a6b
[BWC] fix mixedCluster and rolling upgrades (#775)
This commit fixes mixedCluster and rolling upgrades by spoofing OpenSearch
version 1.0.0 as Legacy version 7.10.2. With this commit an OpenSearch 1.x node
can join a legacy (<= 7.10.2) cluster and rolling upgrades work as expected.
Mixed clusters will not work beyond the duration of the upgrade since shards
cannot be replicated from upgraded nodes to nodes running older versions.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: Shweta Thareja <tharejas@amazon.com>
2021-05-28 00:42:57 -05:00
Rabi Panda 50abf6d066
[CVE] Upgrade dependencies to mitigate CVEs (#657)
This PR upgrade the following dependencies to fix CVEs.

- commons-codec:1.12 (->1.13) apache/commons-codec@48b6157
- ant:1.10.8 (->1.10.9) https://ant.apache.org/security.html
- jackson-databind:2.10.4 (->2.11.0) FasterXML/jackson-databind#2589
- jackson-dataformat-cbor:2.10.4 (->2.11.0) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-28491
- apache-httpclient:4.5.10 (->4.5.13) https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2020-13956
- checkstyle:8.20 (->8.29) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10782
- junit:4.12 (->4.13.1) https://github.com/junit-team/junit4/security/advisories/GHSA-269g-pwp5-87pp
- netty:4.1.49.Final (->4.1.59) https://github.com/netty/netty/security/advisories/GHSA-5mcr-gq6c-3hq2

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-05-18 11:37:24 -07:00
Sooraj Sinha a8b8a96036
Remove URL content from Reindex error response (#630)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-05-07 07:33:16 -07:00
Sooraj Sinha 181ee8a211
Add Remote Reindex SPI extension (#547)
This change extends the remote reindex SPI to allow adding a custom interceptor.
This interceptor can be plugged in to perform any processing on the request or response.

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-05-06 09:38:25 -07:00
Robert Muir 13aa169896
Speedup lang-painless tests (#605)
Painless tests would previously create script engine for every single test method.
Now the tests that need to tweak script engine settings create a class
level fixture (BeforeClass/AfterClass) that is used across all the test
methods in that suite.

RegexLimitTests was split into two suites (limit=1 and limit=2) rather
than dynamically applying different settings.

C2 compiler is no longer needed for tests to be fast, instead tests run
faster with C1 only as expected, like the rest of the unit tests.

Signed-off-by: Robert Muir <rmuir@apache.org>
2021-04-27 13:36:19 -05:00