Commit Graph

55101 Commits

Author SHA1 Message Date
Nicholas Walter Knize 93cce825eb
[BWC] Re-enable bwc testing after 1.0.1 version bump
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-26 07:11:53 -05:00
Nick Knize 7ed024e2c8
Add 1.0.1 revision (#1152)
This commit stages the branch to the next 1.0.1 patch release. BWC testing needs
this even if the next revision is never actually released.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-25 23:11:43 -05:00
Nicholas Walter Knize 26a4db6282
[BWC] Temporarily disable bwc testing while bumping 1.0.1
BWC tests are temporarily disabled to unblock main PRs while bumping 1.0.1

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-25 22:33:59 -05:00
Nicholas Walter Knize 33cabe2e45
[Bug] Change 1.0.0 version check in PluginInfo
PluginInfo should use .onOrAfter(Version.V_1_1_0) instead of
.after(Version.V_1_0_0) for the new custom folder name for plugin feature.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-25 16:30:11 -05:00
Nick Knize 5ae00456a0
Upgrade to Lucene 8.9 (#1080)
This commit upgrades to the official lucene 8.9 release

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-20 11:28:06 -05:00
Nick Knize ff7e7904ca
[DEPRECATE] SimpleFS in favor of NIOFS (#1073)
Lucene 9 removes support for SimpleFS File System format. This commit deprecates
the SimpleFS format in favor of NIOFS.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-08-19 17:56:55 -05:00
Sorabh 93a621f54e
Fix failure in SearchCancellationIT.testMSearchChildReqCancellationWithHybridTimeout (#1103)
In some cases as one shared with issue #1099, the maxConcurrentSearchRequests was chosen as 0 which
will compute the final value during execution of the request based on processor counts. When this
computed value is less than number of search request in msearch request, it will execute all the
requests in multiple iterations causing the failure since test will only wait for one such
iteration. Hence setting the maxConcurrentSearchRequests explicitly to number of search requests
being added in the test to ensure correct behavior

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>
2021-08-17 14:10:31 -04:00
Miki 911c13a395
Add themed logo to README (#988)
Signed-off-by: Miki <mehranb@amazon.com>
2021-08-14 13:19:06 -04:00
Vacha 072ccda932
Support for bwc tests for plugins (#1051)
* Support for bwc tests for plugins

Signed-off-by: Vacha <vachshah@amazon.com>

* Adding support for restart upgrades for plugins bwc

Signed-off-by: Vacha <vachshah@amazon.com>
2021-08-12 13:28:08 -07:00
kartg af6fbc77eb
Improving the Grok circular reference check to prevent stack overflow (#1079)
This change refactors the circular reference check in the Grok processor class
to use a formal depth-first traversal. It also includes a logic update to
prevent a stack overflow in one scenario and a check for malformed patterns.
This bugfix addresses CVE-2021-22144.

Signed-off-by: Kartik Ganesh <85275476+kartg@users.noreply.github.com>
2021-08-12 12:52:02 -04:00
Sorabh 9b6e621452
Part 1: Support for cancel_after_timeinterval parameter in search and msearch request (#986)
* Part 1: Support for cancel_after_timeinterval parameter in search and msearch request

This commit introduces the new request level parameter to configure the timeout interval after which
a search request will be cancelled. For msearch request the parameter is supported both at parent
request and at sub child search requests. If it is provided at parent level and child search request
doesn't have it then the parent level value is set at such child request. The parent level msearch
is not used to cancel the parent request as it may be tricky to come up with correct value in cases
when child search request can have different runtimes

TEST: Added test for ser/de with new parameter

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Part 2: Support for cancel_after_timeinterval parameter in search and msearch request

This commit adds the handling of the new request level parameter and schedule cancellation task. It
also adds a cluster setting to set a global cancellation timeout for search request which will be
used in absence of request level timeout.

TEST: Added new tests in SearchCancellationIT
Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address Review feedback for Part 1

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address review feedback for Part 2

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Update CancellableTask to remove the cancelOnTimeout boolean flag

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Replace search.cancellation.timeout cluster setting with search.enforce_server.timeout.cancellation to control if cluster level cancel_after_time_interval should take precedence over request level cancel_after_time_interval value

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Removing the search.enforce_server.timeout.cancellation cluster setting and just keeping search.cancel_after_time_interval setting with request level parameter taking the precedence.

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

Co-authored-by: Sorabh Hamirwasia <hsorabh@amazon.com>
2021-08-12 11:01:28 -04:00
Daniel Doubrovkine (dB.) 5cd085c713
Removed beta from new issues. (#1071)
Signed-off-by: dblock <dblock@amazon.com>
2021-08-11 00:19:17 -04:00
frotsch c34f9ab75c
Avoid crashing on using the index.lifecycle.name in the API body (#1060)
* Avoid crashing on using the index.lifecycle.name in the API body

Signed-off-by: frotsch <frotsch@mailbox.org>
2021-08-10 11:03:19 -04:00
Bukhtawar Khan fbdc180944
Max scroll limit breach to throw a OpenSearchRejectedExecutionException (#1054)
* Changes the Exception to throw a OpenSearchRejectedExecutionException on max scroll limit breach

Signed-off-by: Bukhtawar Khan bukhtawa@amazon.com
2021-08-10 11:02:04 -04:00
Tianli Feng ca3ced981f
Add support to generate code coverage report with JaCoCo (#971)
* add jacoco plugin

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* config aggregated jacoco report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Skip generating report if no test in subproject

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add jacoco plugin into root project

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* test aggregate code coverage report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Can generate aggregated unit test coverage report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Can generate aggregated test report with source file linked

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Some cleanup, but jacocoReport is not dependson Test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* cleanup

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* get unit test code coverage report

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* change 'enabled == true' to 'enabled'

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* add a comment for selectedsubprojects

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* add tasks to generate code coverage report for unit test and integration test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* fix typo in variable

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Correct the task to get codecoverage for unit test and integtest

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Make code coverage report task run after test

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* apply gradle configuration aciidance api

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* apply jacoco plugin in BuildPlugin in all case

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Put file path list of integration test exec data in a variable

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Apply gradle configuration aviodance api to register task instead of create task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* merge 2 jacocoreport configurations

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Add some comments in gradle script

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Attach code coverage report task to gralde check task

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* add a space

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* get code coverage report after check task on demand and get report in html format on demand

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2021-08-10 07:23:46 -07:00
Rabi Panda e78040aec4
Replace JCenter with Maven Central. (#1057)
On February 3 2021, JFrog [announced](https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/) the shutdown of JCenter. Later on April 27 2021, an update was provided that the repository will only be read only and new package and versions are no longer accepted on JCenter.  This means we should no longer use JCenter for our central artifacts repository.

This change replaces JCenter with Maven Central as per the Gradle recommendation - https://blog.gradle.org/jcenter-shutdown

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-08-09 17:07:43 -07:00
Sven R dcd9cef56c
alt bash path support (#1047)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-06 11:09:29 -04:00
Rabi Panda 7469bf6e26
Include sources and javadoc artifacts while publishing to a Maven repository (#1049)
This change fixes the issue where the sources and javadoc artifacts were not built and included with the publish.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-08-05 16:43:31 -07:00
xuezhou25 7d18eaf3a4
Replaced custom built JNA by official JNA distribution. (#1003)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-08-03 13:52:53 -04:00
Sven R 0fd14e8c3c
alt bash path support (#1013)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-03 09:10:30 -04:00
Bukhtawar Khan 1a87534c20
Prevent /_cat/master from getting tripped by the CB (#1036)
_cat/master is a fundamental API to know the master instance in the cluster. Given RestClusterState is exempted from tripping already, doesn't make sense for RestMasterAction to trip

Signed-off-by: Bukhtawar Khan bukhtawa@amazon.com
2021-08-02 10:15:42 -07:00
Vlad Rozov b7cf1fae41
Introduce RestHandler.Wrapper to help with delegate implementations (#1004)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-30 15:17:27 -04:00
Vlad Rozov ee644bdc40
Add unit test for RestActionListener. Validate that onFailure() sends response even when BytesRestResponse can not be constructed using passed exception. Follow up on #923. (#1024)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-30 14:38:14 -04:00
Yevhen Tienkaiev f46f6950ec
Rank feature - unknown field linear (#983)
Signed-off-by: Yevhen Tienkaiev <hronom@gmail.com>
2021-07-29 14:28:29 -04:00
Vacha 3744f40c28
Excluding missed broken links from link checker (#1010)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-27 18:25:35 -07:00
Vacha 25f0b1e6bd
Validation for official plugins for upgrade tool (#973)
Add validation to check for official plugins during the plugins installation task for the upgrade tool.

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-27 09:01:10 -07:00
Vacha 8edb34a917
Excluding links from link checker (#995)
Excluding all the links from license and notice files to be avoided to be checked by link checker GitHub action.

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-22 14:58:28 -07:00
Vacha 19e54d6624
Allowing custom folder name for plugin installation (#848)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-22 04:28:49 +05:30
Poojita Raj 6bc4ce017a
Replace Elasticsearch docs links in scripts (#994)
Replace the docs links In scripts bin/opensearch-env and config/jvm.options, with OpenSearch docs links.

Signed-off-by: Poojita-Raj <poojiraj@amazon.com>
2021-07-21 14:23:22 -07:00
xuezhou25 e3d86baa5a
Fix opensearch-env always sources the environment from hardcoded file (#875)
distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-07-21 07:51:26 -07:00
afazel 23d604571c
Introduce replaceRoutes() method and 2 new constructors to RestHandler.java (#947)
* Add addRoutesPrefix() method to RestHandler.java

Signed-off-by: Azar Fazel <azar.fazel@gmail.com>
2021-07-15 19:14:15 -07:00
Vlad Rozov 854967ff92
Fail fast when BytesRestResponse ctor throws exception (#923)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-15 14:52:28 -05:00
Himanshu Setia 13a02becba
Restricting logs permissions (#966)
Currently, the permissions for opensearch logs are -rw-r-r-, which gives read access to anyone. This weak permission
structure can lead to leakage of any sensitive information (if published) in the logs. This commit restricts read
access with -rw-r--- permission.

Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-07-15 10:06:45 -07:00
Chang Liu 7241127c1d
Avoid override of routes() in BaseRestHandler to respect the default behavior defined in RestHandler (#889) 2021-07-14 16:22:39 -05:00
Rabi Panda 4906a3c1b0
A CLI tool to assist during an upgrade to OpenSearch. (#846)
This change adds the initial version of a new CLI tool `opensearch-upgrade` as part of the OpenSearch distribution. This tool is meant for assisting during an upgrade from an existing Elasticsearch v7.10.2/v6.8.0 node to OpenSearch. It automates the process of importing existing configurations and installing of core plugins.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-07-13 09:51:02 -07:00
Vacha c7617b03e8
Replacing docs-beta links with /docs (#957)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-13 07:46:05 -07:00
Vacha e17ce53eb7
Adding broken links checker (#877)
* Adding broken links checker

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Adding exclusions for links

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Correcting broken link

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Removing the benchmarks link

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-12 14:07:56 -07:00
Daniel Doubrovkine (dB.) 88f17b1822
Cleanup TESTING and DEVELOPER_GUIDE markdowns (#946) 2021-07-12 15:02:56 -05:00
Abbas Hussain 62523702c1
Update release note for GA 1.0 with new commits and removes #547 (#953)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-07-12 22:27:16 +05:30
Daniel Doubrovkine (dB.) 5d020170ff
Lower build requirement from Java 14+ to Java 11+ (#940)
* Lower build requirement from Java 14+ to Java 11+

Avoid use of -Werror -Xlint:all, which may change significantly across
java releases (new warnings could be added). Instead, just list the
warnings individually.

Workaround JDK 11 compiler bug (JDK-8209058) that only impacts test fixture
code in the build itself.

Signed-off-by: Robert Muir <rmuir@apache.org>

* Disable warning around -source 7 -release 7 for java version checker

The java version checker triggers some default warnings because it
targets java7:

```
> Task :distribution:tools:java-version-checker:compileJava FAILED
warning: [options] source value 7 is obsolete and will be removed in a future release
warning: [options] target value 7 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
error: warnings found and -Werror specified
```

Suppress this warning explicitly for this module.

Signed-off-by: Robert Muir <rmuir@apache.org>

* more java14 -> java11 cleanup

Signed-off-by: Robert Muir <rmuir@apache.org>

Co-authored-by: Robert Muir <rmuir@apache.org>
2021-07-06 11:55:23 -05:00
Sooraj Sinha a428e6f043
Pass interceptor to super constructor (#876)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-07-05 11:13:36 +05:30
Nick Knize 2d1a1e6234
[BUG] Fix cat.health test failures in pre 1.0.0 mixed cluster test (#928)
Fixes the cat.health yaml failures when running in a bwc mixed cluster with
legacy (pre 1.0.0) nodes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-07-02 17:07:15 -05:00
Marc Handalian 25bbdb7219
Update DistributionDownloader to support fetching arm64 bundles. (#929)
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-07-02 17:07:02 -05:00
Rabi Panda f652e95236
Update favicon for OpenSearch (#932)
Signed-off-by: Rabi Panda <pandarab@amazon.com>
2021-07-02 13:40:28 -07:00
Abbas Hussain 4b1f0b0d62
Adding release notes for 1.0.0 (#885)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-07-03 01:27:53 +05:30
shwetathareja 8082604ec2
Refresh OpenSearch nodes version in cluster state after upgrade (#865)
Signed-off-by: Shweta Thareja <tharejas@amazon.com>

Co-authored-by: Shweta Thareja <tharejas@amazon.com>
2021-07-02 09:03:34 -07:00
Tianli Feng 4a1add9481
Add 'tagline' back to MainResponse in server that was removed in PR #427 (#913)
* Add "tagline" field back to "MainResponse" in sever side (not in rest-high-level-client side) that removed in PR #427 .
* Replace with a new tagline "The OpenSearch Project: https://opensearch.org/".
* Turn the tagline into a constant in server/src/main/java/org/opensearch/action/main/MainResponse.java.
2021-07-01 15:30:29 -07:00
Marc Handalian 2335194ebe
Fix Snapshot pattern in DistributionDownloader. (#916)
Snapshots are published with a -latest flag into s3, this updates
the pattern to correctly point to -latest.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-07-01 16:53:17 -05:00
Marc Handalian c91c29e5c2
Remove distribution from main response in compatibility mode (#898)
This Change removes version.distribution when the version.number is
overridden with the cluster setting compatibility.override_main_response_version.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-06-30 17:36:21 -07:00
Vlad Rozov 362f116abe
Replace metadata keys in OpenSearchException during serialization and deserialization (#905)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-06-30 11:39:26 -07:00