Commit Graph

55275 Commits

Author SHA1 Message Date
Tianli Feng 53d60b72b2
Fix defects in code-coverage.gralde to generate code coverage report properly (#1214)
* Refactor the logic to control the format for code coverage report and rename the system property

* Remove outdated code of giving JaCoCo files permission when Java security manager enabled

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2021-09-13 10:56:11 -07:00
Nick Knize aecc7bd005
[Bug] Fix InstallPluginCommand to use proper key signatures (#1233)
The public key has changed since the initial release. This commit fixes the
public key and uses the .sig files that are published to the artifacts site.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-09-10 14:58:31 -07:00
Abbas Hussain 65abe4a45c
Generate release notes for 1.1 (#1230)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-09-09 00:51:53 +05:30
Rabi Panda 3870522228
Run link checker GitHub action on schedule. (#1221)
Most PRs do not add/update links, however sites go down often. This change makes sure that we catch any broken link in the repository and fix it, but at the same time we do not want to block PRs because of some unrelated broken links.

This PR updates the workflow to run everyday at midnight UTC.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-09-08 11:24:19 -07:00
Sai 7e440af9e4
Changes to support retrieval of operations from translog based on specified range (#1210)
* Changes to support retrieval of operations from translog based on specified range

Signed-off-by: Sai Kumar <karanas@amazon.com>

* Addressed CR comments

Signed-off-by: Sai Kumar <karanas@amazon.com>

* Added testcases for internal engine

Signed-off-by: Sai Kumar <karanas@amazon.com>
2021-09-08 20:22:24 +05:30
Andriy Redko 0b773c2cec
Fix org.opensearch.index.reindex.ReindexRestClientSslTests#testClientSucceedsWithCertificateAuthorities - javax.net.ssl.SSLPeerUnverifiedException (#1212)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-08 09:08:05 -04:00
Andriy Redko ddb37dea32
Added Eclipse import instructions to DEVELOPER_GUIDE.md (#1215)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-08 09:05:02 -04:00
Sai a9e1913bd5
Support for translog pruning based on retention leases (#1038)
* Support for translog pruning based on retention leases

Signed-off-by: Sai Kumar <karanas@amazon.com>

* Addressed CR Comments

Signed-off-by: Sai Kumar <karanas@amazon.com>

* Addressed test case issue

Signed-off-by: Sai Kumar <karanas@amazon.com>
2021-09-08 16:23:00 +05:30
Sai 919c5e05aa
Exclude failing links from plugins/modules (#1223)
Signed-off-by: Sai Kumar <karanas@amazon.com>
2021-09-08 13:39:53 +05:30
Sven R 268d93b375
FreeBSD Java support (#1014)
Signed-off-by: hackacad <admin@hackacad.net>
2021-09-07 18:17:58 -04:00
Vacha 83332c8ab6
Kept the original constructor for PluginInfo to maintain bwc (#1206) 2021-09-02 18:09:15 -07:00
Anas Alkouz f37f29c996
Add more instructions how to install/configure git secrets (#1202)
Signed-off-by: Anas Alkouz <aalkouz@amazon.com>

Co-authored-by: Anas Alkouz <aalkouz@amazon.com>
2021-09-02 17:24:12 -04:00
Nick Knize 4b31e1ba04
[Version] Increment main to 2.0 (#1192)
Increment version on main to 2.0.0.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: dblock <dblock@amazon.com>
2021-09-01 22:35:53 -05:00
Abbas Hussain fa8126004c
Upgrade apache commons-compress to 1.21 (#1197)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-09-02 08:35:42 +05:30
Daniel Doubrovkine (dB.) ecd9875e7c
Clarify opensearch.version to not include -SNAPSHOT. (#1186)
Signed-off-by: dblock <dblock@dblock.org>
2021-09-01 21:54:41 -04:00
Daniel Doubrovkine (dB.) 20c6f786ac
Added all icu-project.org websites to the link checker exclusions. (#1201)
Signed-off-by: dblock <dblock@dblock.org>
2021-09-01 21:13:48 -04:00
Andriy Redko 6e199d223b
Restoring alpha/beta/rc version semantics (#1112)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-01 18:22:20 -05:00
Tianli Feng 652cdbd5af
Clarify JDK requirement in the developer guide (#1153)
* Explicitly point out the JDK 8 requirement is for runtime, but not for compiling.
* Clarify the JAVAx_HOME env variables are for the "backwards compatibility test".
* Add explanation on how the backwards compatibility tests get the OpenSearch distributions for a specific version.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2021-09-01 14:15:22 -07:00
Nick Knize 3a7f0762ac
[Bug] Fix mixed cluster support for OpenSearch 2+ (#1191)
The version framework only added support for OpenSearch 1.x bwc with legacy
clusters. This commit adds support for v2.0 which will be the last version with
bwc support for legacy clusters (v7.10)

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-09-01 14:45:46 -05:00
Daniel Doubrovkine (dB.) f298a41e4b
Extract excludes into a file, fix the link checker by adding http://site.icu-project.org/. (#1189) 2021-08-31 18:20:48 -04:00
Andriy Redko f60d093c63
Drop mocksocket & securemock dependencies from sniffer and rest client (no needed) (#1174)
* Drop mocksocket & securemock dependencies from sniffer and rest client (not needed)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Removing .gitignore

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-08-31 14:18:37 -04:00
Minal Shah a43e6cf424
Fixed copyright to OpenSearch (#1175) 2021-08-31 12:55:28 -04:00
Abbas Hussain 91d9922f3b
Reduce iterations to improve test run time (#1168)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-08-30 23:51:52 +05:30
Abbas Hussain 9b4440d1f6
Tune datanode count and shards count to improve test run time (#1170)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-08-30 22:33:31 +05:30
Nicholas Walter Knize 93cce825eb
[BWC] Re-enable bwc testing after 1.0.1 version bump
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-26 07:11:53 -05:00
Nick Knize 7ed024e2c8
Add 1.0.1 revision (#1152)
This commit stages the branch to the next 1.0.1 patch release. BWC testing needs
this even if the next revision is never actually released.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-25 23:11:43 -05:00
Nicholas Walter Knize 26a4db6282
[BWC] Temporarily disable bwc testing while bumping 1.0.1
BWC tests are temporarily disabled to unblock main PRs while bumping 1.0.1

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-25 22:33:59 -05:00
Nicholas Walter Knize 33cabe2e45
[Bug] Change 1.0.0 version check in PluginInfo
PluginInfo should use .onOrAfter(Version.V_1_1_0) instead of
.after(Version.V_1_0_0) for the new custom folder name for plugin feature.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-25 16:30:11 -05:00
Nick Knize 5ae00456a0
Upgrade to Lucene 8.9 (#1080)
This commit upgrades to the official lucene 8.9 release

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-20 11:28:06 -05:00
Nick Knize ff7e7904ca
[DEPRECATE] SimpleFS in favor of NIOFS (#1073)
Lucene 9 removes support for SimpleFS File System format. This commit deprecates
the SimpleFS format in favor of NIOFS.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-19 17:56:55 -05:00
Sorabh 93a621f54e
Fix failure in SearchCancellationIT.testMSearchChildReqCancellationWithHybridTimeout (#1103)
In some cases as one shared with issue #1099, the maxConcurrentSearchRequests was chosen as 0 which
will compute the final value during execution of the request based on processor counts. When this
computed value is less than number of search request in msearch request, it will execute all the
requests in multiple iterations causing the failure since test will only wait for one such
iteration. Hence setting the maxConcurrentSearchRequests explicitly to number of search requests
being added in the test to ensure correct behavior

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>
2021-08-17 14:10:31 -04:00
Miki 911c13a395
Add themed logo to README (#988)
Signed-off-by: Miki <mehranb@amazon.com>
2021-08-14 13:19:06 -04:00
Vacha 072ccda932
Support for bwc tests for plugins (#1051)
* Support for bwc tests for plugins

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding support for restart upgrades for plugins bwc

Signed-off-by: Vacha <vachshah@amazon.com>
2021-08-12 13:28:08 -07:00
kartg af6fbc77eb
Improving the Grok circular reference check to prevent stack overflow (#1079)
This change refactors the circular reference check in the Grok processor class
to use a formal depth-first traversal. It also includes a logic update to
prevent a stack overflow in one scenario and a check for malformed patterns.
This bugfix addresses CVE-2021-22144.

Signed-off-by: Kartik Ganesh <85275476+kartg@users.noreply.github.com>
2021-08-12 12:52:02 -04:00
Sorabh 9b6e621452
Part 1: Support for cancel_after_timeinterval parameter in search and msearch request (#986)
* Part 1: Support for cancel_after_timeinterval parameter in search and msearch request

This commit introduces the new request level parameter to configure the timeout interval after which
a search request will be cancelled. For msearch request the parameter is supported both at parent
request and at sub child search requests. If it is provided at parent level and child search request
doesn't have it then the parent level value is set at such child request. The parent level msearch
is not used to cancel the parent request as it may be tricky to come up with correct value in cases
when child search request can have different runtimes

TEST: Added test for ser/de with new parameter

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Part 2: Support for cancel_after_timeinterval parameter in search and msearch request

This commit adds the handling of the new request level parameter and schedule cancellation task. It
also adds a cluster setting to set a global cancellation timeout for search request which will be
used in absence of request level timeout.

TEST: Added new tests in SearchCancellationIT
Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address Review feedback for Part 1

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address review feedback for Part 2

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Update CancellableTask to remove the cancelOnTimeout boolean flag

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Replace search.cancellation.timeout cluster setting with search.enforce_server.timeout.cancellation to control if cluster level cancel_after_time_interval should take precedence over request level cancel_after_time_interval value

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Removing the search.enforce_server.timeout.cancellation cluster setting and just keeping search.cancel_after_time_interval setting with request level parameter taking the precedence.

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

Co-authored-by: Sorabh Hamirwasia <hsorabh@amazon.com>
2021-08-12 11:01:28 -04:00
Daniel Doubrovkine (dB.) 5cd085c713
Removed beta from new issues. (#1071)
Signed-off-by: dblock <dblock@amazon.com>
2021-08-11 00:19:17 -04:00
frotsch c34f9ab75c
Avoid crashing on using the index.lifecycle.name in the API body (#1060)
* Avoid crashing on using the index.lifecycle.name in the API body

Signed-off-by: frotsch <frotsch@mailbox.org>
2021-08-10 11:03:19 -04:00
Bukhtawar Khan fbdc180944
Max scroll limit breach to throw a OpenSearchRejectedExecutionException (#1054)
* Changes the Exception to throw a OpenSearchRejectedExecutionException on max scroll limit breach

Signed-off-by: Bukhtawar Khan bukhtawa@amazon.com
2021-08-10 11:02:04 -04:00
Tianli Feng ca3ced981f
Add support to generate code coverage report with JaCoCo (#971)
* add jacoco plugin

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* config aggregated jacoco report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Skip generating report if no test in subproject

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add jacoco plugin into root project

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* test aggregate code coverage report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Can generate aggregated unit test coverage report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Can generate aggregated test report with source file linked

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Some cleanup, but jacocoReport is not dependson Test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* cleanup

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* get unit test code coverage report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* change 'enabled == true' to 'enabled'

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* add a comment for selectedsubprojects

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* add tasks to generate code coverage report for unit test and integration test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* fix typo in variable

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Correct the task to get codecoverage for unit test and integtest

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Make code coverage report task run after test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* apply gradle configuration aciidance api

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* apply jacoco plugin in BuildPlugin in all case

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Put file path list of integration test exec data in a variable

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Apply gradle configuration aviodance api to register task instead of create task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* merge 2 jacocoreport configurations

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add some comments in gradle script

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Attach code coverage report task to gralde check task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* add a space

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* get code coverage report after check task on demand and get report in html format on demand

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2021-08-10 07:23:46 -07:00
Rabi Panda e78040aec4
Replace JCenter with Maven Central. (#1057)
On February 3 2021, JFrog [announced](https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/) the shutdown of JCenter. Later on April 27 2021, an update was provided that the repository will only be read only and new package and versions are no longer accepted on JCenter.  This means we should no longer use JCenter for our central artifacts repository.

This change replaces JCenter with Maven Central as per the Gradle recommendation - https://blog.gradle.org/jcenter-shutdown

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-08-09 17:07:43 -07:00
Sven R dcd9cef56c
alt bash path support (#1047)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-06 11:09:29 -04:00
Rabi Panda 7469bf6e26
Include sources and javadoc artifacts while publishing to a Maven repository (#1049)
This change fixes the issue where the sources and javadoc artifacts were not built and included with the publish.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-08-05 16:43:31 -07:00
xuezhou25 7d18eaf3a4
Replaced custom built JNA by official JNA distribution. (#1003)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-08-03 13:52:53 -04:00
Sven R 0fd14e8c3c
alt bash path support (#1013)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-03 09:10:30 -04:00
Bukhtawar Khan 1a87534c20
Prevent /_cat/master from getting tripped by the CB (#1036)
_cat/master is a fundamental API to know the master instance in the cluster. Given RestClusterState is exempted from tripping already, doesn't make sense for RestMasterAction to trip

Signed-off-by: Bukhtawar Khan bukhtawa@amazon.com
2021-08-02 10:15:42 -07:00
Vlad Rozov b7cf1fae41
Introduce RestHandler.Wrapper to help with delegate implementations (#1004)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-30 15:17:27 -04:00
Vlad Rozov ee644bdc40
Add unit test for RestActionListener. Validate that onFailure() sends response even when BytesRestResponse can not be constructed using passed exception. Follow up on #923. (#1024)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-30 14:38:14 -04:00
Yevhen Tienkaiev f46f6950ec
Rank feature - unknown field linear (#983)
Signed-off-by: Yevhen Tienkaiev <hronom@gmail.com>
2021-07-29 14:28:29 -04:00
Vacha 3744f40c28
Excluding missed broken links from link checker (#1010)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-27 18:25:35 -07:00
Vacha 25f0b1e6bd
Validation for official plugins for upgrade tool (#973)
Add validation to check for official plugins during the plugins installation task for the upgrade tool.

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-27 09:01:10 -07:00