Commit Graph

316 Commits

Author SHA1 Message Date
Nick Knize 5358502107
[Upgrade] Lucene-9.2.0-snapshot-ba8c3a8 (#3416)
Upgrades to latest snapshot of lucene 9.2.0 in preparation for GA release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-05-20 16:29:51 -05:00
Nick Knize d86c88fe59
[Refactor] XContentType to parse Accept or Content-Type headers (#3077)
Refactors XContentType.fromMediaTypeOrFormat to fromMediaType so Accept headers
and Content-Type headers can be parsed separately. This helps in reusing the
same parse logic in for REST Versioning API support.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-28 09:58:26 -05:00
Andriy Redko 3b7e654757
Remove JavaVersion, use builtin Runtime.Version to deal with runtime versions (#3006)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-04-20 17:14:02 -04:00
Andriy Redko d61d170332
Removed java11 source folders since JDK-11 is the baseline now (#2898)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-04-18 10:16:22 -07:00
Owais Kazi 3c5d997a76
Added a new line linter (#2875)
* Added linter to add new line

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed new lines

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Ignore empty files

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Updated DEVELOPER GUIDE

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Renamed workflow file

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed failing tests

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-04-13 14:14:18 -04:00
dependabot[bot] 725b5ca004
Bump joni from 2.1.41 to 2.1.43 in /libs/grok (#2832)
* Bump joni from 2.1.41 to 2.1.43 in /libs/grok

Bumps [joni](https://github.com/jruby/joni) from 2.1.41 to 2.1.43.
- [Release notes](https://github.com/jruby/joni/releases)
- [Commits](https://github.com/jruby/joni/compare/joni-2.1.41...joni-2.1.43)

---
updated-dependencies:
- dependency-name: org.jruby.joni:joni
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
2022-04-11 11:48:56 -04:00
dependabot[bot] 6103010e14
Bump jcodings from 1.0.44 to 1.0.57 in /libs/grok (#2736)
* Bump jcodings from 1.0.44 to 1.0.57 in /libs/grok

Bumps [jcodings](https://github.com/jruby/jcodings) from 1.0.44 to 1.0.57.
- [Release notes](https://github.com/jruby/jcodings/releases)
- [Commits](https://github.com/jruby/jcodings/compare/jcodings-1.0.44...jcodings-1.0.57)

---
updated-dependencies:
- dependency-name: org.jruby.jcodings:jcodings
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Fix forbiddenApisCheck for libs/grok.

This change removes the ignoreMissingClasses block of the thirdPartyAudit check.
This class is not missing, so the check causes failures.

Signed-off-by: Marc Handalian <handalm@amazon.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
2022-04-04 17:40:08 -07:00
Andriy Redko d8a1ba6912
[CVE-2020-36518] Update jackson-databind to 2.13.2.2 (#2599)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-03-29 12:24:37 -04:00
Peter Nied d43235c5cf
Move Jackson-databind to 2.13.2 (#2548)
Resolves CVE-2020-36518

Signed-off-by: Peter Nied <petern@amazon.com>
2022-03-21 18:44:13 -05:00
Nick Knize 05a5819243
[Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30 (#2487)
* [Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30

Upgrades from Lucene 9.0.0 to 9.1.0-snapshot-ea989fe8f30 in preparation for
9.1.0 GA.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Add spanishplural token filter

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix KNOWN_TOKENIZERS

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-16 15:47:25 -04:00
Kartik fb9e150076
Refactoring gated and ref-counted interfaces and their implementations (#2396)
* Reducing duplication in plugins around ref-counted releasable classes

Both AmazonEc2Reference and AmazonS3Reference duplicate the same logic - a subclass of AbstractRefCounted that also implements Releasable. This change centralizes this paradigm into a AbstractRefCountedReleasable class and supports both clients via generics. It also updates all fetching implementations to use the get() method instead of client()

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Introduce Reference classes for the Closeable and AutoCloseable interfaces

These classes allow you to wrap a reference instance with an onClose runnable that is executed when close() is invoked. Two separate classes are needed because the close() signatures for the two interfaces are different. This change takes the first step to have implementing classes extend from these generic superclasses, before attempting to remove the subclasses entirely. The get() method is also replaced throughout the code base.

Note that there is also a separate Releasable interface that has a similar access pattern, but is implemented separately. This is used in AbstractRefCountedReleasable introduced in a prior commit

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* More improvements and refactoring

* Functionality around one-way gating is now moved to a dedicated class - OneWayGate. This replaces duplicate functionality throughout the code.
* The two *Reference classes have been renamed to Gated* since that better represents their functionality
* The AbstractRefCountedReleasable has been improved to no longer be abstract by accepting the shutdown hook. This removes the need for the inner class in ReleasableBytesReference, and further simplifies the plugin subclasses (these could probably be removed entirely).
* Finally, unit tests have been added for some classes

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Added tests for GatedCloseable

Also updated the license information in GatedAutoCloseableTests

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Fixing license information in new files

Signed-off-by: Kartik Ganesh <gkart@amazon.com>

* Added unit tests for RefCountedReleasable

Signed-off-by: Kartik Ganesh <gkart@amazon.com>
2022-03-09 12:21:09 -08:00
Daniel Doubrovkine (dB.) 044f53630a
Set target and source compatibility to 11, required by Lucene 9. (#2407)
* Set target and source compatibility to 11, required by Lucene 9.

Signed-off-by: dblock <dblock@dblock.org>

* Uncomment commented code in #2321 for killing child processes that uses JDK9+ ProcessInfo.

Signed-off-by: dblock <dblock@dblock.org>

* Set distribution checker target JDK compatibility to 11.

Signed-off-by: dblock <dblock@dblock.org>

* Supress processing warnings.

Signed-off-by: dblock <dblock@dblock.org>
2022-03-08 14:48:51 -05:00
aponb be64af2f90
Replace exclusionary words whitelist and blacklist in the places that won't impact backwards compatibility (#2178)
* Replace the exclusionary word whitelist with allowlist, and blacklist with denylist, in code commet and internal variable/method/class/package name.

Signed-off-by: Andreas <apre@gmx.at>
2022-03-04 16:34:09 -08:00
dependabot[bot] 8398b8de84
Bump joni from 2.1.29 to 2.1.41 in /libs/grok (#2121)
* Bump joni from 2.1.29 to 2.1.41 in /libs/grok

Bumps [joni](https://github.com/jruby/joni) from 2.1.29 to 2.1.41.
- [Release notes](https://github.com/jruby/joni/releases)
- [Commits](https://github.com/jruby/joni/compare/joni-2.1.29...joni-2.1.41)

---
updated-dependencies:
- dependency-name: org.jruby.joni:joni
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Adding missing classes

Signed-off-by: Vacha Shah <vachshah@amazon.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Vacha Shah <vachshah@amazon.com>
2022-02-22 10:22:45 -08:00
dependabot[bot] 33c5a1ecb7
Bump jopt-simple from 5.0.2 to 5.0.4 in /libs/cli (#2120)
* Bump jopt-simple from 5.0.2 to 5.0.4 in /libs/cli

Bumps [jopt-simple](https://github.com/jopt-simple/jopt-simple) from 5.0.2 to 5.0.4.
- [Release notes](https://github.com/jopt-simple/jopt-simple/releases)
- [Commits](https://github.com/jopt-simple/jopt-simple/compare/jopt-simple-5.0.2...jopt-simple-5.0.4)

---
updated-dependencies:
- dependency-name: net.sf.jopt-simple:jopt-simple
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
2022-02-18 15:27:51 -08:00
Rishikesh Pasham 1568407c36
Upgrading Jackson-Databind version (#1982)
* Upgrading Jackson-Databind version

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding jackson-databind version using getProperty method

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-01-26 20:44:22 -08:00
Andriy Redko 385b268bc0
Update Mockito to 4.2.x (#1830)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-03 12:00:45 -05:00
Fabio Buso 5966cc03bf
Fix JavaDoc typo in XContentBuilder (#1739)
Signed-off-by: Fabio Buso <buso.fabio@gmail.com>
2021-12-16 09:57:38 -05:00
Xue Zhou f54cc382d5
remove codeQL warning about implicit narrowing conversion in compound assignment (#1403)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-11-02 13:49:25 -07:00
Owais Kazi 8394f541bc
Run spotless and exclude checkstyle on libs module (#1428)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-26 09:45:26 -05:00
Andriy Redko 3779576c51
Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests (#1358)
* Modernize and consolidate JDKs usage across all stages of the build. Use JDK-17 as bundled JDK distribution to run tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Using -Djava.security.egd=file:/dev/urandom explicitly for cli tests

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-13 17:25:48 -04:00
Andriy Redko e9635d6bfe
Replace securemock with mock-maker (test support), update Mockito to 3.12.4 (#1332)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-10 14:18:54 -04:00
Andriy Redko cdbc84f09d
Update Jackson to 2.12.5 (#1247)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-21 18:33:20 -04:00
Andriy Redko b6c8bdf872
Drop mocksocket in favour of custom security manager checks (tests only) (#1205)
* Drop mocksocket in favour of custom security manager checks (tests only)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Slightly relaxed host checks to allow all local addresses

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-16 17:21:47 -04:00
kartg af6fbc77eb
Improving the Grok circular reference check to prevent stack overflow (#1079)
This change refactors the circular reference check in the Grok processor class
to use a formal depth-first traversal. It also includes a logic update to
prevent a stack overflow in one scenario and a check for malformed patterns.
This bugfix addresses CVE-2021-22144.

Signed-off-by: Kartik Ganesh <85275476+kartg@users.noreply.github.com>
2021-08-12 12:52:02 -04:00
Sven R dcd9cef56c
alt bash path support (#1047)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-06 11:09:29 -04:00
Tianli Feng 110cef7882
Fix resource leak issues suggested by Amazon CodeGuru (#816)
* Address a kind of issue suggested by Amazon CodeGuru Reviewer:

* Add try-with-resources block to automatically close the resources after using to avoid resource leak, in `SymbolicLinkPreservingTarIT`, `LicenseAnalyzer`, `SymbolicLinkPreservingUntarTransform`, `ConcurrentSeqNoVersioningIT` in `VersionProperties`, `GeoFilterIT`, `XContentHelper`,  `Json` and `IndexShard` class

* Add try-finally block to close the resources after using to avoid resource leak, in `ServerChannelContext` class.

* Add try-catch block to close the resources when exception occurs in `FsBlobContainer` class (when XContentFactory.xContentType throws an exception).

* Close resources when assertion error occurs, in `ServerChannelContext` class.
2021-06-15 09:31:36 -07:00
Rabi Panda 50abf6d066
[CVE] Upgrade dependencies to mitigate CVEs (#657)
This PR upgrade the following dependencies to fix CVEs.

- commons-codec:1.12 (->1.13) apache/commons-codec@48b6157
- ant:1.10.8 (->1.10.9) https://ant.apache.org/security.html
- jackson-databind:2.10.4 (->2.11.0) FasterXML/jackson-databind#2589
- jackson-dataformat-cbor:2.10.4 (->2.11.0) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-28491
- apache-httpclient:4.5.10 (->4.5.13) https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2020-13956
- checkstyle:8.20 (->8.29) https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10782
- junit:4.12 (->4.13.1) https://github.com/junit-team/junit4/security/advisories/GHSA-269g-pwp5-87pp
- netty:4.1.49.Final (->4.1.59) https://github.com/netty/netty/security/advisories/GHSA-5mcr-gq6c-3hq2

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-05-18 11:37:24 -07:00
Daniel Doubrovkine (dB.) 6d1e6a0720
Converted all .asciidoc to .md. (#658)
Signed-off-by: dblock <dblock@amazon.com>
2021-05-05 15:04:14 -04:00
Rabi Panda d89e721da1
Remove the references to xpack and elastic in tests. (#516)
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-04-09 15:21:58 -07:00
Nick Knize 9168f1fb43
[License] Add SPDX and OpenSearch Modification license header (#509)
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-09 14:28:18 -05:00
Rabi Panda 8bba6603da [Rename] Replace more instances of Elasticsearch with OpenSearch. (#432)
This commit replaces more replaceable instances of Elasticsearch with OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 7051167c83 [Rename] remaining elasticsearch pass 1 (#416)
This commit refactors instances of 'elasticsearch' with opensearch everywhere
except references to issues, and other places needed to test compatibility with
old elasticsearch clusters.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda 1bdfbb4ef1 [Rename] Fix imports in the libs module. (#385)
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 5b46a05702 [Rename] remaining packages and resources in test/fixture (#364)
This commit refactors the remaining o.e.index and o.e.test packages in the
test/fixtures module. References throughout the codebase are also refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize 5bef7cec87 [Rename] env,geo,http,indices,ingest,plugin,upgrade packages from test framework (#363)
This commit refactors the following test framework packages:

* o.e.env
* o.e.geo
* o.e.http
* o.e.indices
* o.e.ingest
* o.e.plugin
* o.e.upgrades

to the o.opensearch namespace. All references throughout the test codebase have
been refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize 12f3233fa5 [Rename] framework/test/o.e.action,bootstrap,cli,client,cluster,common (#361)
This commit refactors the following test packages from the o.e namespace:

* o.e.action
* o.e.bootstrap
* o.e.cli
* o.e.client
* o.e.cluster
* o.e.common

to the o.opensearch namespace. Any references throughout the codebase are also
refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Rabi Panda 0aa52bd2e8 [Rename] refactor the libs/x-content module. (#354)
Refactor the code in the `libs/x-content` module and any references to those in the entire code base. The refactoring is done as part of the renaming to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 972d8ea920 [Rename] refactor the libs/core module. (#350)
Refactor the code in the `libs/core` module and any references to those in the entire code base. The refactoring is done as part of the renaming to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 50c3251d36 [Rename] o.e.common.settings (#336)
This commit refactors o.e.common.settings package to the
o.opensearch.common.setttings namespace. All references throughout the codebase
are refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize 3fc453eace [Rename] o.e.common.util (#337)
This commit refactors the o.e.common.util package to the
o.opensearch.common.util namespace. All references throughout the codebase have
been refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize b118646bef [Rename] o.e.common.xcontent (#339)
This commit refactors the o.e.common.xcontent package to
o.opensearch.common.xcontent. All references throughout the codebase have been
refactored.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-03-21 20:56:34 -05:00
Nick Knize c4565adc9d [Rename] o.e.common.geo, hash, io (#317)
This commit refactors the following packages:

* o.e.common.geo
* o.e.common.hash
* o.e.common.io

into the o.opensearch.common namespace. All references throughout the codebase
have been refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 7fd65d1d9e [Rename] refactor the libs/nio module. (#318)
Refactor the `libs/nio` module to rename `org.elasticsearch.nio` to `org.opensearch.nio`.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 73441879f1 [Rename] o.e.common.cache,collect,component,compress,document (#309)
This commit refactors the following:

* o.e.common.cache
* o.e.common.collect
* o.e.common.component
* o.e.common.compress
* o.e.common.document

to the o.opensearch namespace. All references throughout the codebase are also
refactored

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 0f74cbed1c [Rename] o.e.common.blobstore,breaker,bytes (#307)
This commit refactors the following packages:

* o.e.common.blobstore
* o.e.common.breaker
* o.e.common.bytes

to the o.opensearch.common namespace. All references throughout the codebase
have been refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize dafc0510ea [Rename] o.e.common classes (#305)
This commit refactors classes under o.e.common to o.opensearch.common. All
references throughout the codebase have also been refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize fe2b5d6d39 [Rename] o.e.version (#296)
This commit refactors o.e.Version to o.opensearch.Version. This is retained in a
single commit to serve as a reference for re-versioning the opensearch codebase
from legacy 7.10 to 1.0.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 0deb25590d [Rename] server OpenSearch classes (#290)
This commit refactors all OpenSearch classes in the root server package to
o.opensearch. All references throughout the codebase are also refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda ed010c22c8 [Rename] refactor libs/cli module. (#255)
Refactor the `libs/cli` module to rename the package name from `org.elasticsearch.cli` to `org.opensearch.cli` as part of the rename to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00