Commit Graph

2775 Commits

Author SHA1 Message Date
dependabot[bot] 3ce6c025c4
Bump commons-io from 2.7 to 2.11.0 in /plugins/repository-hdfs (#2140)
* Bump commons-io from 2.7 to 2.11.0 in /plugins/repository-hdfs

Bumps commons-io from 2.7 to 2.11.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
2022-02-17 16:00:49 -08:00
Nick Knize 7fe642fda5
[Remove] Type from Search Internals (#2109)
With types deprecation the type support is removed from internal search API
(SearchRequest and QueryShardContext).

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-17 16:17:31 -06:00
Matt Weber cfd96e3e4b
Support unordered non-overlapping intervals (#2103)
* Support unordered non-overlapping intervals

This commit exposes Intervals.unorderedNoOverlaps (LUCENE-8828).

Signed-off-by: Matt Weber <matt@mattweber.org>

* Replace ordered and overlap flags with enum.

- add IntervalMode enum
- deprecate ordered
- bwc between IntervalMode and ordered flag
- update tests
- update rest tests

Signed-off-by: Matt Weber <matt@mattweber.org>

* fix annotated text plugin test

Signed-off-by: Matt Weber <matt@mattweber.org>

* bwc to 2x only

Signed-off-by: Matt Weber <matt@mattweber.org>

* BWC rest tests for ordered flag

Signed-off-by: Matt Weber <matt@mattweber.org>
2022-02-17 17:10:22 -05:00
Andrey Pleskach b9ff91d591
Add proxy settings for GCS repository (#2096)
Added proxy settings for GCS repository.
Security settings:
- gcs.client.*.proxy.username - Proxy user name
- gcs.client.*.proxy.password - Proxy user password

Common settings:
- gcs.client.*.proxy.type - java Proxy.Type names: HTTP, SOCKS. default is DIRECT
- gcs.client.*.proxy.host - Proxy host name
- gcs.client.*.proxy.port - Proxy port

Signed-off-by: Andrey Pleskach <ples@aiven.io>
2022-02-17 11:49:33 -08:00
dependabot[bot] 9689a27b63
Bump com.diffplug.spotless from 5.6.1 to 6.2.0 (#1919)
* Bump com.diffplug.spotless from 5.6.1 to 6.2.0

Bumps com.diffplug.spotless from 5.6.1 to 6.2.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-16 22:08:40 -08:00
Andrey Pleskach 62361ceafc
Add proxy username and password settings for Azure repository (#2098)
Added username/password proxy settings for Azure repository.
Security settings:
- azure.client.*.proxy.username - Proxy user name
- azure.client.*.proxy.password - Proxy user password

Signed-off-by: Andrey Pleskach <ples@aiven.io>
2022-02-15 10:40:47 -08:00
Nick Knize 6428d1a2a6
[Refactor] MapperService to QueryShardContext in valueFetcher (#2027)
* [Refactor] MapperService to QueryShardContext in valueFetcher

QueryShardContext encapsulates MapperService along with other mechanisms useful
for analyzers, highlighters, and readers. This commit refactors the valueFetcher
to use the QueryShardContext as a parameter instead of direct access to the
MapperService. This is an API change. Behavior of the method is unaffected.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* change unnecessary Optional usage

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-08 08:38:20 -05:00
Martin Gaievski 309397520c
Adding support for JDK17 and removing JDK8 (#2025)
* Adding support for JDK17 and removing JDK8

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Merge overlaping PR, bumping min java version to 11

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Removing references to JDK8 from dev guide

Signed-off-by: Martin Gaievski <gaievski@amazon.com>
2022-02-02 20:59:10 -05:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
Nick Knize 0f3b72b7bb
[Deprecate] Setting explicit version on analysis component (#1978)
Lucene 9 removes the ability to define an explicit version on an analysis 
component. The version parameter is deprecated at parse time and a warning is 
issued 
to the user through the deprecation logger.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-01-26 20:46:28 -06:00
Tianli Feng 8b8d04173c
Update protobuf-java to 3.19.3 (#1945)
* Update protobuf-java to 3.19.3

Signed-off-by: Tianli Feng <ftl94@live.com>

* Exclude some API usage violations in the package com.google.protobuf for thirdPartyAudit task to pass

Signed-off-by: Tianli Feng <ftl94@live.com>
2022-01-20 11:05:28 -08:00
Andriy Redko 32f2189686
Update Netty to 4.1.73.Final (#1936)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-18 16:30:18 -08:00
Andriy Redko 385b268bc0
Update Mockito to 4.2.x (#1830)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-03 12:00:45 -05:00
Sarat Vemulapalli db91d2efe9
Upgrading bouncycastle to 1.70 (#1832) 2022-01-03 07:35:38 -05:00
Sarat Vemulapalli 7a97018a92
Updatting Netty to 4.1.72.Final (#1831) 2022-01-03 07:30:52 -05:00
Andriy Redko 65804d25a6
Update to log4j 2.17.1 (#1820)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-28 17:06:42 -05:00
Rabi Panda 7c73ed8d4c
Remove deprecated transport client (#1781)
This commit removes all usage of transport client in OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-28 11:28:51 -06:00
Rabi Panda 24d0c9b818
Remove transport client from tests. (#1809)
Transport client has been deprecated. In this commit, we are removing all its usages from the tests.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-12-27 17:43:18 -06:00
Andriy Redko 5f100c5558
[plugin] repository-azure: add configuration settings for connect/write/response/read timeouts (#1789)
* [plugin] repository-azure: add configuration settings for connect/write/response/read timeouts

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments: renaming connectionXxx to connectXxx

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing code review comments: adding timeout comment

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-22 15:48:47 -06:00
Andrew Ross 96d55966a2
Use try-with-resources with MockLogAppender (#1595)
I previously added a helper that started a MockLogAppender to ensure it
was never added to a Logger before it was started. I subsequently found
the opposite case in RolloverIT.java where the appender was stopped
before it was closed, therefore creating a race where a concurrently
running test in the same JVM could cause a logging failure. This seems
like a really easy mistake to make when writing a test or introduce when
refactoring a test. I've made a change to use try-with-resources to
ensure that proper setup and teardown is done. This should make it much
harder to introduce this particular test bug in the future.
Unfortunately, it did involve touching a lot of files. The changes here
are purely structural to leverage try-with-resources; no testing logic
has been changed.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-12-21 15:00:55 -06:00
Andriy Redko ca27c8fd4f
Update to log4j 2.17.0 (#1771) 2021-12-18 09:36:59 -08:00
Andriy Redko 20692a2ff2
[plugin] repository-azure is not working properly hangs on basic operations (#1740) (#1749)
This commit fixes repository-azure hanging on basic operations. This will be reverted 
once it's fixed upstream in the Azure library.

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-16 15:10:52 -06:00
Nick Knize 5550f8d7e2
[Remove] Analyzer Deprecations (#1741)
This commit removes deprecated analyzer instantiation that is no longer
permitted in OpenSearch 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-16 12:51:49 -06:00
Vacha e66ea2c4f3
Avoid logging duplicate deprecation warnings multiple times (#1660)
* Avoid logging duplicate deprecation warnings multiple times

Signed-off-by: Vacha <vachshah@amazon.com>

* Fixes test failures

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding deprecation logger tests

Signed-off-by: Vacha <vachshah@amazon.com>

* Using ConcurrentHashMap keySet

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-12-15 15:26:44 -08:00
Andriy Redko 6db435412b
Upgrade to log4j 2.16.0 (#1721)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-14 07:34:45 -05:00
Andrew Ross 309649ce8a
Upgrade to logj4 2.15.0 (#1698)
Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-12-10 13:03:41 -08:00
Sarat Vemulapalli e0e6995c4a
Updating Log4j to 2.11.2 (#1696)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2021-12-10 08:03:45 -08:00
Nick Knize b74d71fb74
Remove 6.0.* version constants (#1658)
This PR removes LegacyESVersion.V_6_0_* constants including all pre-release
versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-08 13:37:21 -06:00
Nick Knize 17e7a4962d
Remove Deprecated SimpleFS (#1639)
Lucene 9 removes support for SimpleFS File System format. This PR completely
removes SimpleFS support which was deprecated in a previous PR.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-03 09:57:59 -06:00
Andriy Redko 8bf6fa6b4f
Support Gradle 7 (#1609)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-29 14:34:06 -05:00
Vacha fd87f3a2d7
Upgrading commons-codec in hdfs-fixture and cleaning up dependencies in repository-hdfs (#1603)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-23 13:32:33 -05:00
Andrew Ross d467283475
Start MockLogAppender before adding to static context (#1587)
I observed a test failure with the message
'Attempted to append to non-started appender mock' from an assertion in
`OpenSearchTestCase::after`. I believe this indicates that a
MockLogAppender (which is named "mock") was added as an appender to the
static logging context and some other test in the same JVM happened to
cause a logging statement to hit that appender and cause an error, which
then caused an unrelated test to fail (because they share static state
with the logger). Almost all usages of MockLogAppender start it
immediately after creation. I found a few that did not and fixed those.
I also made a static helper in MockLogAppender to start it upon
creation.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-11-19 13:37:46 -05:00
Vacha bcfb57c06a
Upgrade dependency (#1571)
* Upgrading guava, commons-io and apache-ant dependencies

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding failureaccess since guava needs it

Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-18 13:38:49 -05:00
Vacha c6dd484ce3
Upgrading gson to 2.8.9 (#1541)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-15 14:10:29 -05:00
Ryan Bogan 01d1cb0ce6
Updated links for linkchecker (#1539)
Signed-off-by: Ryan Bogan <rbogan@amazon.com>
2021-11-11 18:24:26 -05:00
Andriy Redko 0e6b0c5b4f
[repository-azure] Update to the latest Azure Storage SDK v12, remove privileged runnable wrapper in favor of access helper (#1521)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-11 12:57:34 -08:00
Xue Zhou f54cc382d5
remove codeQL warning about implicit narrowing conversion in compound assignment (#1403)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-11-02 13:49:25 -07:00
Vacha af6ae752b4
Upgrading dependencies (#1491)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-02 13:47:54 -07:00
Himanshu Setia 681e5548c1
Enabling spotless, disabling checkstyle check on plugins (#1488)
* Enabling spotless, disabling checkstyle on below modules

:plugins:mapper-annotated-text
:plugins:mapper-murmur3
:plugins:mapper-size
:plugins:repository-azure
:plugins:repository-gcs
:plugins:repository-hdfs
:plugins:repository-s3
:plugins:store-smb
:plugins:transport-nio
:qa:die-with-dignity

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Enabling spotless for more plugins

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fixing error in merge conflict

Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-11-01 17:40:06 -07:00
Vacha 389b7dfa18
Upgrading dependencies in hdfs plugin (#1466)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-01 12:58:07 -07:00
Nick Knize 53334b2ce4
Upgrade to Lucene 8.10.1 (#1440)
This commit upgrades to the latest release of lucene 8.10

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-10-28 10:06:53 -05:00
Owais Kazi d02443a265
Run spotless and exclude checkstyle on plugins module (#1417)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-21 20:49:03 -04:00
Andriy Redko acac3cc285
Fixing org.opensearch.repositories.azure.AzureBlobContainerRetriesTests and org.opensearch.action.admin.cluster.node.stats.NodeStatsTests (#1390)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-20 15:27:18 -04:00
Andriy Redko 05dc4bf4b3
Fixing post merge 3rd party audit issues (#1384)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-18 22:06:50 -04:00
Andriy Redko 9612fe80b5
[repository-azure] plugin should use Azure Storage SDK v12 for Java (#1302)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-18 19:48:32 -04:00
Vacha c7f5c90a5f
Upgrading netty version to 4.1.69.Final (#1363)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-10-18 17:30:19 -04:00
Vacha d151082832
Upgrade hadoop dependencies for hdfs plugin (#1335)
* Upgrade hadoop dependencies for hdfs plugin

Signed-off-by: Vacha <vachshah@amazon.com>

* Fixing gradle check failures

Signed-off-by: Vacha <vachshah@amazon.com>

* Upgrading htrace-core4 to 4.1.0

Signed-off-by: Vacha <vachshah@amazon.com>
2021-10-14 14:43:49 -04:00
Andriy Redko 3779576c51
Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests (#1358)
* Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Using -Djava.security.egd=file:/dev/urandom explicitly for cli tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-13 17:25:48 -04:00
Andriy Redko cdbc84f09d
Update Jackson to 2.12.5 (#1247)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-21 18:33:20 -04:00
Andriy Redko b6c8bdf872
Drop mocksocket in favour of custom security manager checks (tests only) (#1205)
* Drop mocksocket in favour of custom security manager checks (tests only)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Slightly relaxed host checks to allow all local addresses

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-16 17:21:47 -04:00