Commit Graph

51842 Commits

Author SHA1 Message Date
Adam Locke 11ae062877
[7.6] [DOCS] Explain flood stage watermark (#57321)
* [7.7] [DOCS] Explain flood stage watermark

* Adding id to fix cross-linking issue.
2020-05-28 19:29:32 -04:00
Nik Everett b9fe10866e
Make global ords terms simpler to understand (backport of #57241) (#57311)
When the `terms` enum operates on non-numeric data it can collect it via
global ordinals. It actually has two separate collection strategies for,
one "dense" and one "remapping". Each of *those* strategies has two
"iteration" strategies that it uses to build buckets, depending on
whether or not we need buckets with `0` docs in them. Previously this
was done with several `null` checks and never really explained. This
change replaces those checks with two `CollectionStrategy` classes which
have good stuff like documentation.
2020-05-28 16:52:35 -04:00
Benjamin Trent 24d605e41e
[ML] fixing GET _ml/inference so size param is respected (#57303) (#57308)
`size` was previously ignored when grabbing full trained model configs. 

closes https://github.com/elastic/elasticsearch/issues/57298
2020-05-28 15:45:26 -04:00
Ryan Ernst 2aeb6ce7ad
Omit translog bwc test before 6.3.0 with default distro (#57266)
The new translog bwc test checks a corruption case before 6.3.0.
However, it needs to restart the old node to reproduce, which does not
currently work given how testclusters works when plugins are installed.
As a workaround, this commit omits creating bwc tests before 6.3.0 only
when the default distribution is used.

fixes #57252
2020-05-28 12:38:30 -07:00
Julie Tibshirani 36e5670f76 Unmute FullClusterRestartIT#testSearch. 2020-05-28 10:55:17 -07:00
Julie Tibshirani 10e1dc199d Revert "Remove unused logic from FieldNamesFieldMapper. (#56834)"
This reverts commit 343fb699a4.
2020-05-28 10:54:10 -07:00
Rene Groeschke 7a4a6360b7
Fix up-to-date checks for precommit related tasks (#57203) (#57291)
* Fix up-to-date checks for precommit related tasks

- Do not use lambdas for doFirst / doLast action declarations as this is not supported by gradle up-to-date check
- Use marker output folder for dependencies license task to make task incremental build compliant

* Tweak formatting
2020-05-28 17:12:17 +02:00
James Rodewig 47024ccb3c
[DOCS] Add verify snapshot repository API docs (#57253) (#57288) 2020-05-28 09:48:04 -04:00
James Rodewig 9277ce6f9e
[DOCS] Adds a doc value field example to Mapper Size docs (#57257) (#57280)
Changes:

* Updates snippet to include doc value field example.
* Fixes a broken link to inline script settings.
2020-05-28 09:26:49 -04:00
Martijn van Groningen 225ccd1cfa
Ensure template exists when creating data stream (#57275)
Backporting #56888 to 7.x branch.

Limit the creation of data streams only for namespaces that have a composable template with a data stream definition.

This way we ensure that mappings/settings have been specified and will be used at data stream creation and data stream rollover.

Also remove `timestamp_field` parameter from create data stream request and
let the create data stream api resolve the timestamp field
from the data stream definition snippet inside a composable template.

Relates to #53100
2020-05-28 15:08:25 +02:00
Rene Groeschke 51158a2d8b
Fix deprecation warning in ThirdpartyAuditTask (#57123) (#57224) 2020-05-28 14:41:42 +02:00
Marios Trivyzas fdac9e99fa
SQL: Fix unecessary evaluation for CASE/IIF (#57159) (#57262)
Previously, `CASE` and `IIF` when translated to painless scripts
(used in GROUP BY, HAVING, WHERE) a custom `caseFunction`
registered in the `InternalSqlScriptUtils` was used. This function
received and array of arbitrary length:
```[condition1, result1, condition2, result2, ... elseResult]```

Painless doesn't know of the context and therefore is evaluating
all conditions and results before invoking the `caseFunction` on them.
As a consequence, erroneous result expressions (i.e. division by 0)
where always evaluated despite of the guarding condition.

Replace the `caseFunction` with painless `<cond> ? <res1> : <res2>`
expressions to properly guard the result expressions and only evaluate
the one for which its guarding condition evaluates to true (or of course
the elseResult).

As a bonus, this approach includes performance benefits since we avoid
unnecessary evaluations of both conditions and result expressions.

Fixes: #49672
(cherry picked from commit 9584b345d89f797bfb658212b928b9812804f02f)
2020-05-28 11:30:14 +02:00
István Zoltán Szabó e1cab4feb4 [DOCS] Puts a link into the loss_function variable description (#56678) 2020-05-28 09:46:11 +02:00
Tim Vernum 408250dcc4
Fix smtp.ssl.trust setting for watcher email (#57268)
The ssl.trust setting for Watcher provides a list of hostnames that
should be automatically trusted for SSL hostname verification. It was
accidentally broken when we added the full ssl.* settings for email
notifications (see #45272)

This commit corrects this, so the setting is once again respected,
as long as none of the other ssl settings are configured for email
notifications.

Resolves: #52153
Backport of: #56090
2020-05-28 17:34:13 +10:00
Ryan Ernst 1b59e9ab22
Move test error reporting to java plugin (#57259)
This commit moves the global hook for reporting failed test cases to the
ElasticsearchJavaPlugin. It should always be applied for all java
projects since the Test class is what emits the failures logged.
2020-05-27 17:41:04 -07:00
Ryan Ernst 97353297dc
Move gradle version check to global build info plugin (#57255)
The gradle version check currently exists in BuildPlugin. However, there
is no reason to check this within every project. Instead, this commit
moves the check to the global build info, which is only applied to the
root project. Additionally, this commit removes the check from buildSrc
because it is not really necessary. The check exists really just for
external plugin authors since we use the gradle wrapper for our own
build.
2020-05-27 17:38:02 -07:00
Ryan Ernst fdb8573413
Convert remaining compilerJavaHome reference 2020-05-27 17:04:04 -07:00
Ryan Ernst beb1d0c338
Remove compiler java version flag (#57237)
This commit removes the compiler.java setting from the build. It was
originally added when Gradle was far behind support for the latest jdk,
but is no longer applicable as we don't have any need to update the
supported compile version before gradle supports the newer version. Note
that the runtime version changing support still exists here, this only
ensures we use the same jdk to compile as we use to run gradle.
2020-05-27 16:33:38 -07:00
Ryan Ernst 3ead0a183b
Upgrade bundled jdk to 14.0.1 (#57233) 2020-05-27 15:08:12 -07:00
David Roberts d139a79ef6
[7.x][ML] Fix monitoring if orphaned anomaly detector persistent tasks exist (#57240)
Since #51888 the ML job stats endpoint has returned entries for
jobs that have a persistent task but not job config. Such
orphaned tasks caused monitoring to fail.

This change ignores any such corrupt jobs for monitoring purposes.

Backport of #57235
2020-05-27 22:59:11 +01:00
Dan Hermann 2738998ebb
Limit _cat/indices test to versions with fix (#57244) (#57256) 2020-05-27 16:57:24 -05:00
James Baiera 3b73ce3112
Fix enrich coordinator to reject documents instead of deadlocking (#56247) (#57179)
This PR removes the blocking call to insert ingest documents into a queue in the
coordinator. It replaces it with an offer call which will throw a rejection exception
in the event that the queue is full. This prevents deadlocks of the write threads
when the queue fills to capacity and there are more than one enrich processors
in a pipeline.
2020-05-27 15:32:13 -04:00
Gordon Brown 2f561084f0
Mute FullClusterRestartIT.testSearch (#57249)
This test fails reliably on upgrades from 6.0.0 and 6.0.1. See #57245 for details.
2020-05-27 13:15:05 -06:00
Nhat Nguyen 5b08eaf90c
Fix trimUnsafeCommits for indices created before 6.2 (#57187)
If an upgraded node is restarted multiple times without flushing a new
index commit, then we will wrongly exclude all commits from the starting
commits. This bug is reproducible with these minimal steps: (1) create
an empty index on 6.1.4 with translog retention disabled, (2) upgrade
the cluster to 7.7.0, (3) restart the upgraded the cluster. The problem
is that with the new translog policy can trim translog without having a
new index commit, while the existing commit still refers to the previous
translog generation.

Closes #57091
2020-05-27 15:08:49 -04:00
James Rodewig a0ca0325fe
[DOCS] Reformat `min_hash` token filter docs (#57181) (#57246)
Changes:

* Rewrites description and adds a Lucene link
* Reformats the configurable parameters as a definition list
* Changes the `Theory` heading to `Using the min_hash token filter for
  similarity search`
* Adds some additional detail to the analyzer example
2020-05-27 15:08:23 -04:00
Lee Hinman c0f732b9f6
[7.x] Rename template V2 classes to ComposableTemplate (#57183) (#57232)
Backports the following commits to 7.x:

    Rename template V2 classes to ComposableTemplate (#57183)
2020-05-27 11:01:59 -06:00
Nik Everett 4d5be7c817
Save memory on numeric sig terms when not top (backport of #56789) (#57221)
This saves memory when running numeric significant terms which are not
at the top level by merging its collection into numeric terms and relying
on the optimization that we made in #55873.
2020-05-27 12:03:28 -04:00
Toby Sutor 2f6089c2b9 [DOCS] Add note about restoring snapshot versions (#55225)
As discussed at https://elastic.slack.com/archives/C0D1XEXEZ/p1586939752242300 is it not possible to restore snapshots taken on newer versions into clusters running lower versions. For example, a snapshot created in a 7.6.0 cluster cannot be restored on a 7.5.0 cluster. This needs to be documented.
2020-05-27 11:40:16 -04:00
James Rodewig 6dbe0022fa
[DOCS] Add redirects for 404 pages (#57161)
Adds several redirects for removed pages that return a
404 response.
2020-05-27 11:15:31 -04:00
James Rodewig f4a673c722
[DOCS] Add get snapshot repo API docs (#57150) (#57218) 2020-05-27 11:12:55 -04:00
Jake Landis baca45978c
[7.x] Remove ability to ingore files from spec validation (#56647) (#56691)
An ignore parameter was originally added to the ValidateJsonAgainstSchemaTask
to allow the build to pass for REST specs that did not properly validate
against the schema.

Since the introduction of this task, all schemas that did not validate have
been fixed to now validate properly.

This commit removes the ability to ignore specific files for validation. This
allows any consumers the assurance that all REST specs validate against the schema.
2020-05-27 10:01:14 -05:00
James Rodewig 5ae650ecfd
[DOCS] Fix clarity of URL snapshot repo docs (#56920) (#57212)
Makes following changes to better clarify docs for read-only URL
snapshot repositories:

* Adds an example snippet for registering a URL repository
* Rewrites the protocols paragraph
* Adds a note to explicitly point out that only URLs using the `ftp`,
  `http`, `http`, and `jar` protocols do not need the `path.repo`
  setting.

Fixes #16280
2020-05-27 10:48:19 -04:00
Przemyslaw Gomulka 0e34b2f42e
SlowLoggers using single logger (#56708)
Slow loggers should use single shared logger as otherwise when index is
deleted the log4j logger will remain reachable (log4j is caching) and
will create a memory leak.

closes https://github.com/elastic/elasticsearch/issues/56171
2020-05-27 16:38:31 +02:00
James Rodewig ff63bda4be
[DOCS] Document index alias swaps are atomic (#55418) (#57209)
Co-authored-by: Tugberk Ugurlu <tugberk@outlook.com>
2020-05-27 10:23:51 -04:00
Samidh fdd378e3fb [Docs] Fix typo in start-watch-service.asciidoc (#57182) 2020-05-27 15:50:31 +02:00
James Rodewig 2c2673d759
[DOCS] Fix remote cluster client node docs (#57176) (#57205)
Changes:

* Documents the `r` role in the _cat/nodes API docs

* Fixes a typo in the node docs

Co-authored-by: Nick Canzoneri <nickcanz@github.com>
2020-05-27 08:58:29 -04:00
AndyHunt66 6760c69783 [DOCS] Fix formatting of create API key API docs (#57138) 2020-05-27 08:34:51 -04:00
István Zoltán Szabó 27f258711a [DOCS] Fixes formatting of admonition paragraph in PUT inference API docs. (#57196) 2020-05-27 13:43:55 +02:00
Christoph Büscher 277c5eaf56 Honor IndicesOptions in HLRC putMapping request (#57118)
Currently, the IndicesOptions set on a High Level rest client PutMappingRequest
are not correctly converted to request parameters. This change adds the missing
conversion and tests.

Closes #57045
2020-05-27 12:00:34 +02:00
Alan Woodward d6b79bcd95 Remove Mapper.updateFieldType() (#57151)
When we had multiple mapping types, an update to a field in one type had to be
propagated to the same field in all other types. This was done using the
Mapper.updateFieldType() method, called at the end of a merge. However, now
that we only have a single type per index, this method is unnecessary and can
be removed.

Relates to #41059
Backport of #56986
2020-05-27 09:21:24 +01:00
Julie Tibshirani 343fb699a4 Remove unused logic from FieldNamesFieldMapper. (#56834)
This logic is no longer used, now that each field mapper handles adding the
`_field_names` fields.
2020-05-26 17:40:36 -07:00
Mark Vieira 3a3efeb9c7
Include 'sourceUrl' column in dependencies report even when empty (#57163)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-05-26 15:06:06 -07:00
Ryan Ernst a2864191d6
Move common repository configuration to java plugin (#57057)
This commit moves the common maven repository configuration to the ES
java plugin. All java projects need this common set of repos. Note that
the Elastic download and maven repos are removed, as they are not
necessary anymore since distribution download was split into the
DistributionDownloadPlugin.
2020-05-26 14:12:29 -07:00
Tal Levy 81060820e9 Fix NormalizerAgg test searcher wrapping (#57171)
The searcher was randomly wrapping its reader as slow, parallel, or filtered.
This was causing casting issues in the normalizer tests. By removing the
wrapping, the problem goes away.

Closes #57164
2020-05-26 13:25:19 -07:00
Jake Landis 41629b7bfd
[7.x] Move license checks back to BuildPlugin (#56975) (#57169) 2020-05-26 15:18:12 -05:00
Benjamin Trent decc6277f9
[ML] allow unran/incomplete forecasts to be deleted for stopped/failed jobs (#57152) (#57172)
If a job is NOT opened, forecasts should be able to be deleted, no matter their state.

This also fixes a bug with expanding forecast IDs. We should check for wildcard `*` and `_all` when expanding the ids

closes https://github.com/elastic/elasticsearch/issues/56419
2020-05-26 15:44:22 -04:00
Nik Everett 0fce2b7713 Fix DateHistogramAggregatorTests.testAsSubAgg
Closes #57168 by using `AggregatorTestCase#newIndexSearcher` in the
`AggregatorTestCase#testCase`. Without that global ordinals will
*sometimes* fail to work.
2020-05-26 15:05:31 -04:00
Mark Vieira 92e127e90d
Mute DateHistogramAggregatorTests.testAsSubAgg
(cherry picked from commit 4d050a7a6438a7d102eeef9e03a7d79565bddab7)
2020-05-26 10:57:22 -07:00
Jake Landis 920677af6f
7.x only REST specification fixes (#56736)
Fixes for the REST specification specific to 7.x

* remove ignore "cat.thread_pool.json" and add the "" as valid option. #55984 deprecated this field since it these params here have no effect on this specific API
* remove ignore "indices.put_mapping.json" by adding the required / in the path to pass validation.
2020-05-26 12:33:57 -05:00
Bogdan Pintea 74b2c8a770 Change error message for comp against fields (#57126)
Change the error message wording for comparisons against fields in
filtering (s/variables/fields).

(cherry picked from commit d9a1cb50940d0a98fd75b9c0123ca6e1d862f65d)
2020-05-26 17:57:51 +02:00