Commit Graph

45653 Commits

Author SHA1 Message Date
Martijn Laarman 85b9dc18a7 fix #35262 define deprecations of API's as a whole and urls (#39063)
* fix #35262 define deprecations of API's as a whole and urls

* document hot threads deprecated paths

* deprecate scroll_id as part of the URL, documented only as part of the body which is a safer behaviour as well

* use version numbers up to patch version

* rest spec parser picks up deprecated paths as paths too

(cherry picked from commit 7e06023e7603b7584bfd9ee4e8a1ccd82c208ce7)
2019-04-23 14:28:36 +02:00
David Roberts d8a2970fa4 [TEST] Mute RemoteClusterServiceTests.testCollectNodes
Due to https://github.com/elastic/elasticsearch/issues/41067
2019-04-23 15:13:01 +01:00
Marios Trivyzas e991175776
SQL: Implement IIF(<cond>, <result1>, <result2>) (#41420)
Implement a more trivial case of the CASE expression which is
expressed as a traditional function with 2 or 3 arguments. e.g.:

IIF(a = 1, 'one', 'many')
IIF(a > 0, 'positive')
Closes: #40917

(cherry picked from commit add02f4f553ad472026dcc1eaa84245a0558a4b0)
2019-04-23 16:31:25 +03:00
Hendrik Muhs 31c609db91 [DOCS] change source and dest to be nested and add a query to the example (#41293)
update source and dest to reflect the changes done in #40396 and add a query to the example.
2019-04-23 13:39:43 +02:00
Jason Tedor 4a288af85f
Avoid concurrent modification in mock log appender (#41424)
It can be the case that while we are setting up expectations that also a
log message is appended. For example, if we are setting up these
expectations after a cluster has formed and messages start being sent
around the cluster. In this case, we would hit a concurrent modification
exception while we are mutating the expectations, and also while the
expectations are being iterated over as a message is appended. This
commit avoids this by using a copy-on-write array list which is safe for
concurrent modification and iteration. Note that another possible
approach here is to use synchronized, but that seems unnecessary since
we don't appear to rely on messages that are sent while we are setting
up expectations. Rather, we are setting up some expectations and some
situation that we think will cause those expectations to be met. Using
copy-on-write array list here is nice since we avoid bottlenecking these
tests on synchronizing these methods.
2019-04-22 21:47:16 -04:00
Melori Arellano e6c24e3d2d [DOCS] Add missing setting skip_unavailable to example
The example to delete a remote cluster is missing the `skip_unavailable` setting which results in an error:
```
        "type": "illegal_argument_exception",
        "reason": "missing required setting [cluster.remote.tiny-test.seeds] for setting [cluster.remote.tiny-test.skip_unavailable]"
```
2019-04-22 16:53:18 -04:00
Ryan Ernst cc867e8a5a
Remove /var/run/elasticsearch from packages (#41102)
The pid dir for both systemd and init.d is already managed by those
respective systems (tmpfiles.d and the init script, respectively). Since
the /var/run dir is often mounted as tmpfs, it does not make sense to
have the elasticsearch pid dir added by the package installation. This
commit removes that empty dir from deb and rpm.
2019-04-22 09:57:51 -07:00
Marios Trivyzas 67d4e399c2
SQL: Implement CASE... WHEN... THEN... ELSE... END (#41349)
Implement the ANSI SQL CASE expression which provides the if/else
functionality common to most programming languages.

The CASE expression can have multiple WHEN branches and becomes a
powerful tool for SQL queries as it can be used in SELECT, WHERE,
GROUP BY, HAVING and ORDER BY clauses.

Closes: #36200
(cherry picked from commit 8b2577406f47ae60d15803058921d128390af0b6)
2019-04-22 19:26:56 +03:00
Dimitris Athanasiou eb2295ac81
[7.1][ML] Refactor autodetect service into its own class (#41378) (#41409)
This also improves aims to improve the corresponding unit tests
with regard to readability and maintainability.
2019-04-22 17:42:13 +03:00
James Rodewig d2a418152d [DOCS] Remove inline callouts for Asciidoctor migration (#41309) 2019-04-22 09:35:22 -04:00
James Rodewig 3c60f967af [DOCS] Remove inline callouts in SQL Command docs for Asciidoctor migration (#41276) 2019-04-22 09:23:13 -04:00
James Rodewig 490ad2af46 [DOCS] Remove inline callouts for Asciidoctor migration (#41266) 2019-04-22 09:09:21 -04:00
Ioannis Kakavas b8d054e73b OpenID Connect realm settings and rest API docs (#40740)
This commit adds the relevant docs for the OpenID Connect
realm settings and the REST APIs that are exposed.
2019-04-22 15:50:18 +03:00
James Rodewig d7cddf8157 [DOCS] Replace nested open block for Asciidoctor migration (#41168)
* [DOCS] Fix nested open blocks for Asciidoctor migration

* [DOCS] Reformat table to definitions
2019-04-22 08:40:02 -04:00
James Rodewig c7a7493804 [DOCS] Remove abbrevtitles for Asciidoctor migration (#41366) 2019-04-22 08:28:38 -04:00
Albert Zaharovits c3e0ae24d3
Fix role mapping DN field wildcards for users with NULL DNs (#41343)
The `DistinguishedNamePredicate`, used for matching users to role mapping
expressions, should handle users with null DNs. But it fails to do so (and this is
a NPE bug), if the role mapping expression contains a lucene regexp or a wildcard.

The fix simplifies `DistinguishedNamePredicate` to not handle null DNs at all, and
instead use the `ExpressionModel#NULL_PREDICATE` for the DN field, just like
any other missing user field.
2019-04-22 10:25:24 +03:00
Jason Tedor 5c40fc9ba5
Remove script engine from X-Pack plugin (#41387)
The X-Pack plugin implements ScriptEngine yet it does not actually
implement any of the methods on the interface, effectively making this a
no-op. This commit removes this interface from the X-Pack plugin.
2019-04-20 08:23:20 -04:00
Jason Tedor 21bf2fe3c4
Reduce security permissions in CCR plugin (#41391)
It looks like these permissions were copy/pasted from another plugin yet
almost none of these permissions are needed for the CCR plugin. This
commit removes all these unneeded permissions from the CCR plugin.
2019-04-20 08:21:59 -04:00
Joe Zack 2b20bd0b8d Fix discovery config in docker-compose docs (#41394)
Today's `docker-compose` docs are missing the `discovery.seed_nodes` config on
one of the nodes. With today's configuration the cluster can still form the
first time it is started, because `cluster.initial_master_nodes` requires both
nodes to bootstrap the cluster which ensures that each discover the other.
However if `es02` is elected master it will remove `es01` from the voting
configuration and then when restarted it will form a cluster on its own without
needing to do any discovery. Meanwhile `es01` doesn't know how to find `es02`
after a restart so will be unable to join this cluster.

This commit fixes this by adding the missing configuration.
2019-04-20 09:33:31 +01:00
Jason Tedor ac7d5e3e9b Fix reference to ignore_above from analyis-icu docs
This commit fixes a reference to the docs for ignore_above from the
analysis-icu plugin docs.
2019-04-19 15:18:43 -07:00
clement-tourriere c80f86e3e4 Add ignore_above in ICUCollationKeywordFieldMapper (#40414)
Add the possibility to use ignore_above parameter in ICUCollationKeywordFieldMapper.

Close #40413
2019-04-19 14:19:35 -07:00
Jason Tedor 38b43c4a56
Limit the number of forks getting Java versions (#41386)
To reduce configuration time, we fork some threads to compute the Java
version for the various configured Javas. However, as the number of
JAVA${N}_HOME variable increases, the current implementation creates as
many threads as there are such variables, which could be more than the
number of physical cores on the machine. It is not likely that we would
see benefits to trying to execute all of these once beyond the number of
physical cores (maybe simultaneous multi-threading helps though, who
knows. Therefore, this commit limits the parallelization here to the
number number of physical cores.
2019-04-19 17:16:19 -04:00
Ryan Ernst c0164cbb63 Only include relevant platform files from modules (#41089)
This commit adds a filter to the files include from modules to only
include platform specific files relevant to the distribution being
built. For example, the deb files on linux would now only include linux
ML binaries, and not windows or macos files.
2019-04-19 11:34:50 -07:00
Jason Tedor 154d40494f
Fix build issue if no specific Java version are set (#41379)
If no Java versions are set then when we size the executor thread pool
we end up with zero threads, which is illegal. This commit avoids that
problem by only starting the executor when needed.
2019-04-19 14:09:10 -04:00
Ryan Ernst bcd0939b61 Add a rule for task dependencies (#41322)
This commit adds a task rule to print the task dependencies of any task.
It only prints the direct dependencies, but makes debugging missing
dependencies a lot easier.
2019-04-19 10:02:27 -07:00
Marios Trivyzas 7a34ba35f7
SQL: Fix bug with optimization of null related conditionals (#41355)
The SimplifyConditional rule is removing NULL literals from those
functions to simplify their evaluation. This happens in the Optimizer
and a new instance of the conditional function is generated. Previously,
the dataType was not set properly (defaulted to DataType.NULL) for
those new instances and since the resolveType() wasn't called again
it resulted in returning always null.

E.g.:

SELECT COALESCE(null, 'foo', null, 'bar')

COALESCE(null, 'foo', null, 'bar')
-----------------
null

This issue was not visible before because the tests always used an alias
for the conditional function which caused the resolveType() to be
called which sets the dataType properly.

E.g.:

SELECT COALESCE(null, 'foo', null, 'bar') as c

c
-----------------
foo

(cherry picked from commit c39980a65dd593363f1d8d1b038b26cb0ce02aaf)
2019-04-19 19:04:32 +03:00
Arlind d989079df5
Update glossary.asciidoc (#41364) 2019-04-19 10:25:07 +02:00
Alpar Torok e1e2568fa3 Add FIPS specific testclusters configuration (#41199)
ClusterFormationTasks auto configured these properties for clusters.
This PR adds FIPS specific configuration across all test clusters from
the main build script to prevent coupling betwwen testclusters and the
build plugin.

Closes #40904
2019-04-19 10:36:54 +03:00
David Turner 0bb15d3dac Allow ops to be blocked after primary promotion (#41360)
Today we assert that there are no operations in flight in this test. However we
will sometimes be in a situation where the operations are blocked, and we
distinguish these cases since #41271 causing the assertion to fail. This commit
addresses this by allowing operations to be blocked sometimes after a primary
promotion.

Fixes #41333.
2019-04-19 07:48:43 +01:00
Alpar Torok 4ef4ed66b9 Convert repository-hdfs to testclusters (#41252)
* Convert repository-hdfs to testclusters

Relates #40862
2019-04-19 09:47:14 +03:00
Alpar Torok 2f8bbce85d Clean up build tool dependencies (#41336)
We are no longer using these dependencies.

Relates to #41061 since the class that seems to be leaking is both part
of Gradle and the logging jar.
2019-04-19 09:22:38 +03:00
Alpar Torok 1f91759bc7 Improove the configuration time if the build (#41251)
This will help with reproduction lines and running tests form IDEs and
other operations that are quick and executed often enough for the
configuration time to matter.

Running Gradle with a FIPS JVM is not supproted, so if the runtime JVM
is the same one, no need to spend time checking for fips support.

Verification of the JAVA<version>_HOME env vars is now async and
parallel so it doesn't block configuration.
2019-04-19 08:58:43 +03:00
Lisa Cawley c03394c236 [DOCS] Fixes deprecation notice in pagerduty action (#41362) 2019-04-18 17:33:07 -07:00
Jim Ferenczi 8f73e1e883 Fix unmapped field handling in the composite aggregation (#41280)
The `composite` aggregation maps unknown fields as numerics, this means that
any `after` value that is set on a query with an unmapped field on some indices
will fail if the provided value is not numeric. This commit changes the default
value source to use keyword instead in order to be able to parse any type of after
values.
2019-04-18 23:08:13 +02:00
Jim Ferenczi 754037b71e Unified highlighter should ignore terms that targets the _id field (#41275)
The `_id` field uses a binary encoding to index terms that is not compatible with
the utf8 automaton that the unified highlighter creates to reanalyze the input.
For these reason this commit ignores terms that target the `_id` field when
`require_field_match` is set to false.

Closes #37525
2019-04-18 22:31:23 +02:00
Jim Ferenczi 068f8ba223 more_like_this query to throw an error if the like fields is not provided (#40632)
With the removal of the `_all` field the `mlt` query cannot infer a field name
to use to analyze the provided (un)like text if the `fields` parameter is not
explicitly set in the query and the `index.query.default_field` is not changed
in the index settings (by default it is set to `*`). For this reason the like text
is ignored and queries are only built from the provided document ids.
This change fixes this bug by throwing an error if the fields option is not set
and the `index.query.default_field` is equals to `*`. The error is thrown only
if like or unlike texts are provided in the query.
2019-04-18 22:30:22 +02:00
Jay Modi 1f5cd410b8
Clean up docs regarding recommended JVM (#41356)
This change clarifies the documentation around the recommended JVM. The
recommended JVM is the bundled JVM. If a user does not use our
recommended JVM we suggest that they use a supported LTS version of the
JVM.

Closes #41132
2019-04-18 14:17:48 -06:00
Costin Leau 5f497d4274 SQL: Predicate diff takes into account all values (#41346)
Fix bug in predicate subtraction that caused the evaluation to be
skipped on the first mismatch instead of evaluating the whole list. In
some cases this caused not only an incorrect result but one that kept on
growing causing the engine to bail

Fix #40835

(cherry picked from commit bd2b33d6eaca616a5acd846204e2d12f905854d4)
2019-04-18 22:44:50 +03:00
Andrei Stefan cfed5d65be SQL: fix *SecurityIT tests by covering edge case scenarios when audit file rolls over at midnight (#41328)
* Handle the scenario where assertLogs() is not called from a test method
but the audit rolling file rolls over.
* Use a local boolean variable instead of the static one to account for
assertBusy() code block possibly being called multiple times and having
different execution paths.

(cherry picked from commit 6f642196cbab90079c610097befc794746170df1)
2019-04-18 21:24:18 +03:00
Mark Vieira 0227ac5690
Fix issue with subproject test task dependencies (#41321) (#41351) 2019-04-18 11:15:34 -07:00
Simon Willnauer 11dc9fe249 Mark searcher as accessed in acquireSearcher (#41335)
This fixes an issue where every N seconds a slow search request is triggered
since the searcher access time is not set unless the shard is idle. This change
moves to a more pro-active approach setting the searcher as accessed all the time.
2019-04-18 19:14:50 +02:00
Nik Everett 6b4cf8f0bd Docs: Revert accidental gcs docs change
I snuck an extra change in my last commit accidentally. This reverts it.
2019-04-18 09:22:07 -04:00
Nik Everett 0a343be90c Docs: Fix deprecation warning in Asciidoctor
Fix a deprecation warning that wasn't rendering correctly in
asciidoctor. This one needed to be explicitly marked as an inline macro
because it is on its own line and it needed to have its text escaped
because it contained a `,`. It also was missing explanitory text for
what the setting was.
2019-04-18 09:18:24 -04:00
Adrien Grand a699cb76a5
Fix javadoc tag. (#41330)
s/returns/return/
2019-04-18 14:41:09 +02:00
Armin Braun 389a13b68e
Mute BulkProcessorRetryIT#testBulkRejectionLoadWithBackoff (#41325) (#41331)
* For #41324
2019-04-18 11:55:28 +02:00
Adrien Grand 86e56590a7 Revert "Disable CcrRetentionLeaseIT#testRetentionLeasesAreNotBeingRenewedAfterRecoveryCompletes."
This reverts commit 343039e200.
2019-04-18 11:31:00 +02:00
Adrien Grand 343039e200 Disable CcrRetentionLeaseIT#testRetentionLeasesAreNotBeingRenewedAfterRecoveryCompletes.
Relates #39331.
2019-04-18 11:29:11 +02:00
Alpar Torok 45c151ff8b Disable composePull only if it exists (#41306)
The task will not be created when docker is not available.
2019-04-18 09:54:01 +03:00
Alpar Torok a4a4259cac Mute failing test
Tracking #41326
2019-04-18 09:26:20 +03:00
Alpar Torok 44a0c468cf Clean up clusters between tests (#41187)
This PR adds additional cleanup when stopping the node.
The data dir is excepted because it gets reused in some tests.
Without this cleanup the number of working dir copies could grew to
exhaust all available disk space.
2019-04-18 08:58:47 +03:00