Commit Graph

113 Commits

Author SHA1 Message Date
Tianli Feng 8f8909fd54
Replace 'whitelist' terminology in Java API (#3350)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-17 13:34:05 -07:00
Tianli Feng 8460bcf2ec
Replace 'master' terminology with 'cluster manager' in 'plugins' directory (#3329)
* Replace 'master' terminology with 'cluster manager' in 'plugins' directory

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Replace master_node with cluster_manager_node in cluster.state API test

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-05-16 08:46:39 -04:00
Owais Kazi 3c5d997a76
Added a new line linter (#2875)
* Added linter to add new line

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed new lines

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Ignore empty files

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Updated DEVELOPER GUIDE

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Renamed workflow file

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed failing tests

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-04-13 14:14:18 -04:00
Wenjun Ruan 0b1f4a2069
Rename `file` to `dir` in Environment (#2730)
* Rename `file` to `dir` in Environment

Signed-off-by: ruanwenjun <wenjun@apache.org>

* Fix compile error

Signed-off-by: ruanwenjun <wenjun@apache.org>

* fix compile error

Signed-off-by: ruanwenjun <wenjun@apache.org>
2022-04-05 09:23:08 -04:00
aponb be64af2f90
Replace exclusionary words whitelist and blacklist in the places that won't impact backwards compatibility (#2178)
* Replace the exclusionary word whitelist with allowlist, and blacklist with denylist, in code commet and internal variable/method/class/package name.

Signed-off-by: Andreas <apre@gmx.at>
2022-03-04 16:34:09 -08:00
Xue Zhou f54cc382d5
remove codeQL warning about implicit narrowing conversion in compound assignment (#1403)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-11-02 13:49:25 -07:00
Owais Kazi d02443a265
Run spotless and exclude checkstyle on plugins module (#1417)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-21 20:49:03 -04:00
Andriy Redko b6c8bdf872
Drop mocksocket in favour of custom security manager checks (tests only) (#1205)
* Drop mocksocket in favour of custom security manager checks (tests only)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Slightly relaxed host checks to allow all local addresses

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-09-16 17:21:47 -04:00
Sven R dcd9cef56c
alt bash path support (#1047)
Signed-off-by: hackacad <admin@hackacad.net>
2021-08-06 11:09:29 -04:00
Vacha c7617b03e8
Replacing docs-beta links with /docs (#957)
Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-13 07:46:05 -07:00
Vacha e17ce53eb7
Adding broken links checker (#877)
* Adding broken links checker

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Adding exclusions for links

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Correcting broken link

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Removing the benchmarks link

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2021-07-12 14:07:56 -07:00
Rabi Panda 3fede8be3c
Rename the distribution used in test clusters. (#603)
For test clusters, we are using the archive(zip, tar), so we rename the distribution accordingly.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-04-22 14:21:32 -07:00
Nick Knize 9168f1fb43
[License] Add SPDX and OpenSearch Modification license header (#509)
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-04-09 14:28:18 -05:00
Rabi Panda 2dca3462f2
Fix stragglers from renaming to OpenSearch work. (#483)
This commit fixes more instances where we missed renaming to OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-04-05 11:51:20 -07:00
Rabi Panda 2e3055c9e2
Fix more failing tests as a result of renaming (#457)
This commit fixes some more renaming issues and as a result fixes the failing tests,

* :qa:logging-config:test 
* :example-plugins:painless-whitelist:yamlRestTest
* :modules:reindex:test

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-24 09:33:05 -07:00
Rabi Panda 597b52992d [Rename] File names replace elasticsearch with opensearch. (#419)
This commit renames several files that contain the name elasticsearch and replace that with opensearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda eddfe6760d [Rename] Fix issues for gradle precommit task. (#418)
Fix miscellaneous issues identified during `gradle precommit`. These issues are the side effects of the renaming to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Harold Wang 82f9ff93cb [Rename] plugins (#193)
* [Rename] plugins (#193)

This PR refactors files under "plugins" folders part of the Elasticsearch to OpenSearch renaming effort.

Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 584efd7970 [Rename] modules/lang-painless (#210)
Refactor lang-painless module as part rename to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 3eee5183d1 [Rename] server/rest (#229)
This commit refactors the `server/rest` package as part of the Elasticsearch to OpenSearch renaming.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize ccceb381db [Rename] ElasticsearchException class in server module (#165)
This commit refactors the ElasticsearchException class located in the server module
to OpenSearchException. References and usages throughout the rest of the
codebase are fully refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 38e9c9750a [PURIFY] Remove the AuthorizationEnginePlugin from examples. (#26)
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:09 -06:00
Rabi Panda c856534394 [PURIFY] Remove remaining x-pack license. (#25)
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:09 -06:00
Nick Knize a0b91cb230 Cleanup build script to exclude security-authorization-engine (#8) (#8)
* Cleanup build-scan, remove publish scan to elastic server

* Cleanup build script to exclude security-authorization-engine which test has dependency on xpack

* Cleanup build script to exclude security-authorization-engine which test has dependency on xpack

Co-authored-by: Huan Jiang <huanji@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:06 -06:00
Nick Knize 3a52e9ddc1 [PURIFY] update build.gradle files to ensure build completes; gradle check fails (#7)
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:06 -06:00
Rene Groeschke bdd7347bbf
Merge test runner task into RestIntegTest (7.x backport) (#60600)
* Merge test runner task into RestIntegTest (#60261)
* Merge test runner task into RestIntegTest
* Reorganizing Standalone runner and RestIntegTest task
* Rework general test task configuration and extension
* Fix merge issues
* use former 7.x common test configuration
2020-08-04 14:46:32 +02:00
Jake Landis 6ce30bea08
[7.x] Convert most OSS plugins from integTest to [yaml | java]RestTest or internalClusterTest (#59444) (#60343)
For all OSS plugins (except repository-* and discovery-*) integTest
task is now a no-op and all of the tests are now executed via a test,
yamlRestTest, javaRestTest, or internalClusterTest.

This commit does NOT convert the discovery-* and repository-* since they
are bit more complex then the rest of tests and this PR is large enough.
Those plugins will be addressed in a future PR(s).

This commit also fixes a minor issue that did not copy the rest api
for projects that only had YAML TEST tests.

related: #56841
2020-07-29 13:06:13 -05:00
Jake Landis 604c6dd528
7.x - Create plugin for yamlTest task (#56841) (#59090)
This commit creates a new Gradle plugin to provide a separate task name
and source set for running YAML based REST tests. The only project
converted to use the new plugin in this PR is distribution/archives/integ-test-zip.
For which the testing has been moved to :rest-api-spec since it makes the most
sense and it avoids a small but awkward change to the distribution plugin.

The remaining cases in modules, plugins, and x-pack will be handled in followups.

This plugin is distinctly different from the plugin introduced in #55896 since
the YAML REST tests are intended to be black box tests over HTTP. As such they
should not (by default) have access to the classpath for that which they are testing.

The YAML based REST tests will be moved to separate source sets (yamlRestTest).
The which source is the target for the test resources is dependent on if this
new plugin is applied. If it is not applied, it will default to the test source
set.

Further, this introduces a breaking change for plugin developers that
use the YAML testing framework. They will now need to either use the new source set
and matching task, or configure the rest resources to use the old "test" source set that
matches the old integTest task. (The former should be preferred).

As part of this change (which is also breaking for plugin developers) the
rest resources plugin has been removed from the build plugin and now requires
either explicit application or application via the new YAML REST test plugin.

Plugin developers should be able to fix the breaking changes to the YAML tests
by adding apply plugin: 'elasticsearch.yaml-rest-test' and moving the YAML tests
under a yamlRestTest folder (instead of test)
2020-07-06 14:16:26 -05:00
Rene Groeschke 01e9126588
Remove deprecated usage of testCompile configuration (#57921) (#58083)
* Remove usage of deprecated testCompile configuration
* Replace testCompile usage by testImplementation
* Make testImplementation non transitive by default (as we did for testCompile)
* Update CONTRIBUTING about using testImplementation for test dependencies
* Fail on testCompile configuration usage
2020-06-14 22:30:44 +02:00
Rene Groeschke 28920a45f1
Improvement usage of gradle task avoidance api (#56627) (#56981)
Use gradle task avoidance api wherever it is possible as a drop in replacement in the es build
2020-05-25 09:37:33 +02:00
Nik Everett b98b260048
Merge significant_terms into the terms package (backport of #56699) (#56715)
This merges the code for the `significant_terms` agg into the package
for the code for the `terms` agg. They are *super* entangled already,
this mostly just admits that to ourselves.

Precondition for the terms work in #56487
2020-05-13 17:36:21 -04:00
William Brafford 49e30b15a2
Deprecate disabling basic-license features (#54816) (#55405)
We believe there's no longer a need to be able to disable basic-license
features completely using the "xpack.*.enabled" settings. If users don't
want to use those features, they simply don't need to use them. Having
such features always available lets us build more complex features that
assume basic-license features are present.

This commit deprecates settings of the form "xpack.*.enabled" for
basic-license features, excluding "security", which is a special case.
It also removes deprecated settings from integration tests and unit
tests where they're not directly relevant; e.g. monitoring and ILM are
no longer disabled in many integration tests.
2020-04-17 15:04:17 -04:00
Jason Tedor f3a0018175
Update link to JDK 14 compiler bug
This commit updates the link to the JDK 14 compiler bug that we have
found. At the time that we committed the workaround, we had a submission
ID, but not yet the public bug URL. This commit adds the public bug URL.
2020-04-07 06:26:14 -04:00
Jason Tedor f2590b9984
Workaround JDK 14 compiler bug (#54689)
This commit workarounds a bug in the JDK 14 compiler. It is choking on a
method reference, so we substitute a lambda expression instead. The JDK
bug ID is 9064309.
2020-04-02 19:45:52 -04:00
Jason Tedor 5fcda57b37
Rename MetaData to Metadata in all of the places (#54519)
This is a simple naming change PR, to fix the fact that "metadata" is a
single English word, and for too long we have not followed general
naming conventions for it. We are also not consistent about it, for
example, METADATA instead of META_DATA if we were trying to be
consistent with MetaData (although METADATA is correct when considered
in the context of "metadata"). This was a simple find and replace across
the code base, only taking a few minutes to fix this naming issue
forever.
2020-03-31 17:24:38 -04:00
Martijn van Groningen 4b4fbc160d
Refactor AliasOrIndex abstraction. (#54394)
Backport of #53982

In order to prepare the `AliasOrIndex` abstraction for the introduction of data streams,
the abstraction needs to be made more flexible, because currently it really can be only
an alias or an index.

* Renamed `AliasOrIndex` to `IndexAbstraction`.
* Introduced a `IndexAbstraction.Type` enum to indicate what a `IndexAbstraction` instance is.
* Replaced the `isAlias()` method that returns a boolean with the `getType()` method that returns the new Type enum.
* Moved `getWriteIndex()` up from the `IndexAbstraction.Alias` to the `IndexAbstraction` interface.
* Moved `getAliasName()` up from the `IndexAbstraction.Alias` to the `IndexAbstraction` interface and renamed it to `getName()`.
* Removed unnecessary casting to `IndexAbstraction.Alias` by just checking the `getType()` method.

Relates to #53100
2020-03-30 10:12:16 +02:00
Alan Woodward 71b703edd1 Rename AtomicFieldData to LeafFieldData (#53554)
This conforms with lucene's LeafReader naming convention, and
matches other per-segment structures in elasticsearch.
2020-03-17 12:30:12 +00:00
Jay Modi 3edadfefd0 RestHandlers declare handled routes (#52123)
This commit changes how RestHandlers are registered with the
RestController so that a RestHandler no longer needs to register itself
with the RestController. Instead the RestHandler interface has new
methods which when called provide information about the routes
(method and path combinations) that are handled by the handler
including any deprecated and/or replaced combinations.

This change also makes the publication of RestHandlers safe since they
no longer publish a reference to themselves within their constructors.

Closes #51622

Co-authored-by: Jason Tedor <jason@tedor.me>

Backport of #51950
2020-02-09 22:48:32 -07:00
Mark Vieira f86de2a9cb
Always test against default distribution when in a FIPS JVM (#51273) (#51333) 2020-01-23 14:54:57 -08:00
Mark Vieira c08c282c0e
Revert "Always test against default distribution when in a FIPS JVM (#51273)"
This reverts commit 0169498711.
This reverts commit c5a032b594.
2020-01-22 12:15:57 -08:00
Mark Vieira c5a032b594
Always test against default distribution when in a FIPS JVM (#51273)
(cherry picked from commit e34d7fdaf7b511627c64a9e16805fd82f980b8c6)
2020-01-22 11:30:25 -08:00
Nik Everett f6c89b4599
Move test of custom sig heuristic to plugin (#50891) (#51067)
This moves the testing of custom significance heuristic plugins from an
`ESIntegTestCase` to an example plugin. This is *much* more "real" and
can be used as an example for anyone that needs to actually build such a
plugin. The old test had testing concerns and the example all jumbled
together.
2020-01-16 14:49:12 -05:00
Stuart Tettemer 689df1f28f
Scripting: ScriptFactory not required by compile (#50344) (#50392)
Avoid backwards incompatible changes for 8.x and 7.6 by removing type
restriction on compile and Factory.  Factories may optionally implement
ScriptFactory.  If so, then they can indicate determinism and thus
cacheability.

**Backport**

Relates: #49466
2019-12-19 12:50:25 -07:00
Stuart Tettemer 17cda5b2c0
Scripting: Groundwork for caching script results (#49895) (#49944)
In order to cache script results in the query shard cache, we need to
check if scripts are deterministic.  This change adds a default method
to the script factories, `isResultDeterministic() -> false` which is
used by the `QueryShardContext`.

Script results were never cached and that does not change here.  Future
changes will implement this method based on whether the results of the
scripts are deterministic or not and therefore cacheable.

Refs: #49466

**Backport**
2019-12-06 15:08:05 -07:00
Stuart Tettemer 426c7a5e8f
Scripting: add available languages & contexts API (#49652) (#49815)
Adds `GET /_script_language` to support Kibana dynamic scripting
language selection.

Response contains whether `inline` and/or `stored` scripts are
enabled as determined by the `script.allowed_types` settings.

For each scripting language registered, such as `painless`,
`expression`, `mustache` or custom, available contexts for the language
are included as determined by the `script.allowed_contexts` setting.

Response format:
```
{
  "types_allowed": [
    "inline",
    "stored"
  ],
  "language_contexts": [
    {
      "language": "expression",
      "contexts": [
        "aggregation_selector",
        "aggs"
        ...
      ]
    },
    {
      "language": "painless",
      "contexts": [
        "aggregation_selector",
        "aggs",
        "aggs_combine",
        ...
      ]
    }
...
  ]
}
```

Fixes: #49463 

**Backport**
2019-12-04 16:18:22 -07:00
Rory Hunter c46a0e8708
Apply 2-space indent to all gradle scripts (#49071)
Backport of #48849. Update `.editorconfig` to make the Java settings the
default for all files, and then apply a 2-space indent to all `*.gradle`
files. Then reformat all the files.
2019-11-14 11:01:23 +00:00
Mark Vieira 6ab4645f4e
[7.x] Introduce type-safe and consistent pattern for handling build globals (#48818)
This commit introduces a consistent, and type-safe manner for handling
global build parameters through out our build logic. Primarily this
replaces the existing usages of extra properties with static accessors.
It also introduces and explicit API for initialization and mutation of
any such parameters, as well as better error handling for uninitialized
or eager access of parameter values.

Closes #42042
2019-11-01 11:33:11 -07:00
Ryan Ernst f32692208e
Add explanations to script score queries (#46693) (#47548)
While function scores using scripts do allow explanations, they are only
creatable with an expert plugin. This commit improves the situation for
the newer script score query by adding the ability to set the
explanation from the script itself.

To set the explanation, a user would check for `explanation != null` to
indicate an explanation is needed, and then call
`explanation.set("some description")`.
2019-10-03 21:05:05 -07:00
Luca Cavanna c31cddf27e
Update the schema for the REST API specification (#42346)
* Update the REST API specification

This patch updates the REST API spefication in JSON files to better encode deprecated entities,
to improve specification of URL paths, and to open up the schema for future extensions.

Notably, it changes the `paths` from a list of strings to a list of objects, where each
particular object encodes all the information for this particular path: the `parts` and the `methods`.

Among the benefits of this approach is eg. encoding the difference between using the `PUT` and `POST`
methods in the Index API, to either use a specific document ID, or let Elasticsearch generate one.

Also `documentation` becomes an object that supports an `url` and also a `description` which is a
new field.

* Adapt YAML runner to new REST API specification format

The logic for choosing the path to use when running tests has been
simplified, as a consequence of the path parts being listed under each
path in the spec. The special case for create and index has been removed.

Also the parsing code has been hardened so that errors are thrown earlier
when the structure of the spec differs from what expected, and their
error messages should be more helpful.
2019-08-16 14:40:00 +02:00
Yogesh Gaikwad 471d940c44
Refactor cluster privileges and cluster permission (#45265) (#45442)
The current implementations make it difficult for
adding new privileges (example: a cluster privilege which is
more than cluster action-based and not exposed to the security
administrator). On the high level, we would like our cluster privilege
either:
- a named cluster privilege
  This corresponds to `cluster` field from the role descriptor
- or a configurable cluster privilege
  This corresponds to the `global` field from the role-descriptor and
allows a security administrator to configure them.

Some of the responsibilities like the merging of action based cluster privileges
are now pushed at cluster permission level. How to implement the predicate
(using Automaton) is being now enforced by cluster permission.

`ClusterPermission` helps in enforcing the cluster level access either by
performing checks against cluster action and optionally against a request.
It is a collection of one or more permission checks where if any of the checks
allow access then the permission allows access to a cluster action.

Implementations of cluster privilege must be able to provide information
regarding the predicates to the cluster permission so that can be enforced.
This is enforced by making implementations of cluster privilege aware of
cluster permission builder and provide a way to specify how the permission is
to be built for a given privilege.

This commit renames `ConditionalClusterPrivilege` to `ConfigurableClusterPrivilege`.
`ConfigurableClusterPrivilege` is a renderable cluster privilege exposed
as a `global` field in role descriptor.

Other than this there is a requirement where we would want to know if a cluster
permission is implied by another cluster-permission (`has-privileges`).
This is helpful in addressing queries related to privileges for a user.
This is not just simply checking of cluster permissions since we do not
have access to runtime information (like request object).
This refactoring does not try to address those scenarios.

Relates #44048
2019-08-13 09:06:18 +10:00