Commit Graph

55033 Commits

Author SHA1 Message Date
xuezhou25 7d18eaf3a4
Replaced custom built JNA by official JNA distribution. (#1003)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-08-03 13:52:53 -04:00
Sven R 0fd14e8c3c
alt bash path support (#1013)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-03 09:10:30 -04:00
Bukhtawar Khan 1a87534c20
Prevent /_cat/master from getting tripped by the CB (#1036)
_cat/master is a fundamental API to know the master instance in the cluster. Given RestClusterState is exempted from tripping already, doesn't make sense for RestMasterAction to trip

Signed-off-by: Bukhtawar Khan bukhtawa@amazon.com
2021-08-02 10:15:42 -07:00
Vlad Rozov b7cf1fae41
Introduce RestHandler.Wrapper to help with delegate implementations (#1004)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-30 15:17:27 -04:00
Vlad Rozov ee644bdc40
Add unit test for RestActionListener. Validate that onFailure() sends response even when BytesRestResponse can not be constructed using passed exception. Follow up on #923. (#1024)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-30 14:38:14 -04:00
Yevhen Tienkaiev f46f6950ec
Rank feature - unknown field linear (#983)
Signed-off-by: Yevhen Tienkaiev <hronom@gmail.com>
2021-07-29 14:28:29 -04:00
Vacha 3744f40c28
Excluding missed broken links from link checker (#1010)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-27 18:25:35 -07:00
Vacha 25f0b1e6bd
Validation for official plugins for upgrade tool (#973)
Add validation to check for official plugins during the plugins installation task for the upgrade tool.

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-27 09:01:10 -07:00
Vacha 8edb34a917
Excluding links from link checker (#995)
Excluding all the links from license and notice files to be avoided to be checked by link checker GitHub action.

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-22 14:58:28 -07:00
Vacha 19e54d6624
Allowing custom folder name for plugin installation (#848)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-22 04:28:49 +05:30
Poojita Raj 6bc4ce017a
Replace Elasticsearch docs links in scripts (#994)
Replace the docs links In scripts bin/opensearch-env and config/jvm.options, with OpenSearch docs links.

Signed-off-by: Poojita-Raj <poojiraj@amazon.com>
2021-07-21 14:23:22 -07:00
xuezhou25 e3d86baa5a
Fix opensearch-env always sources the environment from hardcoded file (#875)
distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-07-21 07:51:26 -07:00
afazel 23d604571c
Introduce replaceRoutes() method and 2 new constructors to RestHandler.java (#947)
* Add addRoutesPrefix() method to RestHandler.java

Signed-off-by: Azar Fazel <azar.fazel@gmail.com>
2021-07-15 19:14:15 -07:00
Vlad Rozov 854967ff92
Fail fast when BytesRestResponse ctor throws exception (#923)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-07-15 14:52:28 -05:00
Himanshu Setia 13a02becba
Restricting logs permissions (#966)
Currently, the permissions for opensearch logs are -rw-r-r-, which gives read access to anyone. This weak permission
structure can lead to leakage of any sensitive information (if published) in the logs. This commit restricts read
access with -rw-r--- permission.

Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-07-15 10:06:45 -07:00
Chang Liu 7241127c1d
Avoid override of routes() in BaseRestHandler to respect the default behavior defined in RestHandler (#889) 2021-07-14 16:22:39 -05:00
Rabi Panda 4906a3c1b0
A CLI tool to assist during an upgrade to OpenSearch. (#846)
This change adds the initial version of a new CLI tool `opensearch-upgrade` as part of the OpenSearch distribution. This tool is meant for assisting during an upgrade from an existing Elasticsearch v7.10.2/v6.8.0 node to OpenSearch. It automates the process of importing existing configurations and installing of core plugins.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-07-13 09:51:02 -07:00
Vacha c7617b03e8
Replacing docs-beta links with /docs (#957)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-13 07:46:05 -07:00
Vacha e17ce53eb7
Adding broken links checker (#877)
* Adding broken links checker

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Adding exclusions for links

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Correcting broken link

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Removing the benchmarks link

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-12 14:07:56 -07:00
Daniel Doubrovkine (dB.) 88f17b1822
Cleanup TESTING and DEVELOPER_GUIDE markdowns (#946) 2021-07-12 15:02:56 -05:00
Abbas Hussain 62523702c1
Update release note for GA 1.0 with new commits and removes #547 (#953)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-07-12 22:27:16 +05:30
Daniel Doubrovkine (dB.) 5d020170ff
Lower build requirement from Java 14+ to Java 11+ (#940)
* Lower build requirement from Java 14+ to Java 11+

Avoid use of -Werror -Xlint:all, which may change significantly across
java releases (new warnings could be added). Instead, just list the
warnings individually.

Workaround JDK 11 compiler bug (JDK-8209058) that only impacts test fixture
code in the build itself.

Signed-off-by: Robert Muir <rmuir@apache.org>

* Disable warning around -source 7 -release 7 for java version checker

The java version checker triggers some default warnings because it
targets java7:

```
> Task :distribution:tools:java-version-checker:compileJava FAILED
warning: [options] source value 7 is obsolete and will be removed in a future release
warning: [options] target value 7 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
error: warnings found and -Werror specified
```

Suppress this warning explicitly for this module.

Signed-off-by: Robert Muir <rmuir@apache.org>

* more java14 -> java11 cleanup

Signed-off-by: Robert Muir <rmuir@apache.org>

Co-authored-by: Robert Muir <rmuir@apache.org>
2021-07-06 11:55:23 -05:00
Sooraj Sinha a428e6f043
Pass interceptor to super constructor (#876)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
2021-07-05 11:13:36 +05:30
Nick Knize 2d1a1e6234
[BUG] Fix cat.health test failures in pre 1.0.0 mixed cluster test (#928)
Fixes the cat.health yaml failures when running in a bwc mixed cluster with
legacy (pre 1.0.0) nodes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-07-02 17:07:15 -05:00
Marc Handalian 25bbdb7219
Update DistributionDownloader to support fetching arm64 bundles. (#929)
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-07-02 17:07:02 -05:00
Rabi Panda f652e95236
Update favicon for OpenSearch (#932)
Signed-off-by: Rabi Panda <pandarab@amazon.com>
2021-07-02 13:40:28 -07:00
Abbas Hussain 4b1f0b0d62
Adding release notes for 1.0.0 (#885)
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
2021-07-03 01:27:53 +05:30
shwetathareja 8082604ec2
Refresh OpenSearch nodes version in cluster state after upgrade (#865)
Signed-off-by: Shweta Thareja <tharejas@amazon.com>

Co-authored-by: Shweta Thareja <tharejas@amazon.com>
2021-07-02 09:03:34 -07:00
Tianli Feng 4a1add9481
Add 'tagline' back to MainResponse in server that was removed in PR #427 (#913)
* Add "tagline" field back to "MainResponse" in sever side (not in rest-high-level-client side) that removed in PR #427 .
* Replace with a new tagline "The OpenSearch Project: https://opensearch.org/".
* Turn the tagline into a constant in server/src/main/java/org/opensearch/action/main/MainResponse.java.
2021-07-01 15:30:29 -07:00
Marc Handalian 2335194ebe
Fix Snapshot pattern in DistributionDownloader. (#916)
Snapshots are published with a -latest flag into s3, this updates
the pattern to correctly point to -latest.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-07-01 16:53:17 -05:00
Marc Handalian c91c29e5c2
Remove distribution from main response in compatibility mode (#898)
This Change removes version.distribution when the version.number is
overridden with the cluster setting compatibility.override_main_response_version.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-06-30 17:36:21 -07:00
Vlad Rozov 362f116abe
Replace metadata keys in OpenSearchException during serialization and deserialization (#905)
Signed-off-by: Vlad Rozov <vrozov@users.noreply.github.com>
2021-06-30 11:39:26 -07:00
Marc Handalian 06228bc25b
Update DistributionDownloader to fetch snapshots and staging bundles. (#904)
This change updates DistributionDownloader by registering a second
repository containing release-candidate bundles.  This repository will
only be checked if the release repository does not contain the requested
version.

This change also updates the snapshot repository to point to a new
snapshot repository.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-06-30 10:29:07 -07:00
Nick Knize ef1cbd9e3d
Version bump for 1.1 release (#772)
Bumps the minor to 1.1 for the 1.0 release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-06-28 09:58:27 -05:00
Nick Knize 0317b604f6
[BUG] Fix versioning issues discovered through version bump (#884)
This commit fixes up the version framework to ensure a smooth version bump
across releases.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-06-25 16:53:06 -05:00
Tianli Feng 18625952a9
update external library 'pdfbox' version to 2.0.24 to reduce vulnerability (#883) 2021-06-25 13:18:15 -07:00
Ralph Ursprung 0079fde645
docker build: use OSS `log4j2.properties` (#878)
with Elasticsearch there were two flavours of the docker build: one for
the OSS build and one with X-Pack. accordingly, there were two
`log4j2.properties` files for the two cases and the `build.gradle`
picked up the correct one depending on the build.

commit 83e87f7e54 removed all non-OSS functionality from the build but
with that it also removed the logic which so far copied over the OSS
version of `log4j2.properties`.

what the diff doesn't show properly here is that `log4j2.properties` has
been deleted and `oss/log4j2.properties` has been moved into its place
without any changes to it.

this fixes #868

Signed-off-by: Ralph Ursprung <Ralph.Ursprung@avaloq.com>
2021-06-24 10:16:17 -07:00
Marc Handalian c1250c963d
Add cluster setting to spoof version number returned from MainResponse (#847)
This change adds a new cluster setting "compatibility.override_main_response_version"
that when enabled spoofs the version.number returned from MainResponse
for REST clients expecting legacy version 7.10.2.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-06-22 10:42:23 -07:00
Rabi Panda ca564fd04f
Enable adding experimental features through sandbox modules (#691)
OpenSearch uses plugins to add new features. A plugin can be included in the distribution by default (as part of *modules* directory) or can be installed optionally from a plugin repository.

This change provides a separate space called *sandbox* inside OpenSearch for the community to easily experiment new ideas and innovate. Ideally, this is where an experimental feature will reside before it can be promoted to the *modules* directory. All the plugins in this module will only be included in the **snapshot distributions**. During assembling distributions, we will check if the following two conditions are met to include the sandbox modules,

* the distribution is a snapshot i.e. the build system property `build.snapshot` is set to true. We use this because it will prevent accidental bundling of these modules in a release distribution.
* the `enable.sandbox` system property is set to true. By default it is set to true. The purpose of adding this extra flag is that we can exclude the modules from snapshots if needed. For example, we may want to run performance tests on snapshots without the sandbox modules.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-06-22 11:45:18 -05:00
Marc Handalian c7e60b87f9
Add URL for lucene snapshots (#858)
Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-06-22 11:40:53 -05:00
Daniel Doubrovkine (dB.) 6018a5d6d3
Updated READMEs on releasing, maintaining, admins and security. (#853)
* Updated READMEs on releasing, maintaining, admins and security.

Signed-off-by: dblock <dblock@amazon.com>

* Ensuing -> following.

Signed-off-by: dblock <dblock@amazon.com>
2021-06-17 18:03:11 -04:00
itiyamas 11babe7782
Decouple throttling limits for new and old indices. (#778)
* Decouple throttling limits for new and old indices.

Signed-off-by: Itiyama <itiyamas@amazon.com>

* Precommit fixes.

Signed-off-by: Itiyama <itiyamas@amazon.com>

* Review comments.

Signed-off-by: Itiyama <itiyamas@amazon.com>

* Review comments and test fix.

Signed-off-by: Itiyama <itiyamas@amazon.com>

* Checkstyle fixes.

Signed-off-by: Itiyama <itiyamas@amazon.com>

* Review comments.

Signed-off-by: Itiyama <itiyamas@amazon.com>
2021-06-17 10:58:50 +05:30
jayesh hathila 3cd4e7ff54
Verbose plugin not found exception (#849)
* Add Plugin name for verbose Plugin not found exception
* Make the plugin loading failure exception more verbose
* Throw Opensearch in place of RuntimeException for plugin load failure
* Nit fix, added ... to make logging standout

Signed-off-by: Jayesh Hathila <sharma.jayesh52@gmail.com>
2021-06-15 15:12:01 -04:00
Tianli Feng 110cef7882
Fix resource leak issues suggested by Amazon CodeGuru (#816)
* Address a kind of issue suggested by Amazon CodeGuru Reviewer:

* Add try-with-resources block to automatically close the resources after using to avoid resource leak, in `SymbolicLinkPreservingTarIT`, `LicenseAnalyzer`, `SymbolicLinkPreservingUntarTransform`, `ConcurrentSeqNoVersioningIT` in `VersionProperties`, `GeoFilterIT`, `XContentHelper`,  `Json` and `IndexShard` class

* Add try-finally block to close the resources after using to avoid resource leak, in `ServerChannelContext` class.

* Add try-catch block to close the resources when exception occurs in `FsBlobContainer` class (when XContentFactory.xContentType throws an exception).

* Close resources when assertion error occurs, in `ServerChannelContext` class.
2021-06-15 09:31:36 -07:00
Marc Handalian dff274414d
Add Snapshot maven repository (#829)
Signed-off-by: Marc Handalian <handalm@amazon.com>
2021-06-10 14:07:48 -07:00
Jesper Olsson e7812cd882
#699/first impression (#712)
* #699 Add logo

Signed-off-by: Jesper Olsson <software@jesperolsson.se>

* #699 Describe OpenSearch

Signed-off-by: Jesper Olsson <software@jesperolsson.se>

* #699 Reorganize content

Signed-off-by: Jesper Olsson <software@jesperolsson.se>

* #699 Simplify helping out

Signed-off-by: Jesper Olsson <software@jesperolsson.se>

* #699 Clarify forum

Signed-off-by: Jesper Olsson <software@jesperolsson.se>

* #699 Remove history

Signed-off-by: Jesper Olsson <software@jesperolsson.se>
2021-06-08 12:00:47 -04:00
Gregor Zurowski 4fff57014f
Remove `client/sniffer` from Javadoc exemption list (#818)
Signed-off-by: Gregor Zurowski <gregor@zurowski.org>
2021-06-04 17:15:08 -07:00
Peter Nied 74f29c7727
Fix arm architecture translation issue (#809)
Found when attempting to build on an `arm64` machine when I recieved an error message below.  Root cause is that string equality in java cannot be done with the `==` sign.

```
unknown architecture [arm64] for jdk [provisioned_runtime], must be one of [aarch64, x64]
```

Signed-off-by: Peter Nied <petern@amazon.com>
2021-06-04 12:26:46 -04:00
Gregor Zurowski 0f9060761c
Fix Javadoc errors in `client/sniffer` (#802)
* Adds a gradle plugin to validate missing javadocs

Use `./gradlew missingJavadoc` to validate missing javadocs.
Currently this task fails because several modules are missing
appropriate javadocs. Once added, this should pass.
Also, precommit PomValidation check currently fails with missing Javadoc
plugin, that needs to be fixed -
https://github.com/opensearch-project/OpenSearch/issues/449
Thus keeping this in a separate feature branch.

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fix Javadoc errors in client/sniffer module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

* Add package info to client/sniffer module

Signed-off-by: Gregor Zurowski <gregor@zurowski.org>

Co-authored-by: Himanshu Setia <setiah@amazon.com>
2021-06-03 17:08:16 -07:00
Daniel Doubrovkine (dB.) 6a8e303591
Removed pre-alpha notes. (#815)
Signed-off-by: dblock <dblock@amazon.com>
2021-06-03 14:50:59 -07:00