Commit Graph

53059 Commits

Author SHA1 Message Date
Armin Braun 7ae9dc2092
Unify Stream Copy Buffer Usage (#56078) (#60608)
We have various ways of copying between two streams and handling thread-local
buffers throughout the codebase. This commit unifies a number of them and
removes buffer allocations in many spots.
2020-08-04 09:54:52 +02:00
Yang Wang 54aaadade7
API key name should always be required for creation (#59836) (#60636)
The name is now required when creating or granting API keys.
2020-08-04 13:28:47 +10:00
Tim Vernum c58e32bb27
Improve assertion failure when error is not empty (#60572)
This commit changes TokenAuthIntegTests so all occurrences of

    assertThat(x.size(), equalTo(0));

become

    assertThat(x, empty());

This means that the assertion failure message will include the
contents of the list (`x`) instead of just its size, which
facilitates easier failure diagnosis.

Relates: #56903
Backport of: #60496
2020-08-04 11:05:18 +10:00
Russ Cam 667309e6a6 [DOCS] Fix list dangling indices documentation (#60099)
This commit fixes the list dangling indices response.
The dangling_indices array is an array of objects
that represent aggregated dangling index information

(cherry picked from commit 24c72d4e71c95f2d7690090933e0657152f6af9b)
2020-08-04 10:32:00 +10:00
Julie Tibshirani f99584c6f3
Avoid reloading _source for every inner hit. (#60632)
Previously if an inner_hits block required _ source, we would reload and parse
the root document's source for every hit. This PR adds a shared SourceLookup to
the inner hits context that allows inner hits to reuse parsed source if it's
already available. This matches our approach for sharing the root document ID.

Relates to #32818.
2020-08-03 17:12:27 -07:00
debadair 80584d266d
[DOCS] Update link to ILM tutorial (#60557) (#60624) 2020-08-03 13:04:24 -07:00
debadair e9ac195756
[DOCS] Add info about why we removed test fw docs (#60346) (#60558)
* [DOCS] Add info about why we removed test fw docs

* Apply suggestions from code review

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-08-03 12:05:35 -07:00
Jake Landis bcb9d06bb6
[7.x] Cleanup xpack build.gradle (#60554) (#60603)
This commit does three things:
* Removes all Copyright/license headers for the build.gradle files under x-pack. (implicit Apache license)
* Removes evaluationDependsOn(xpackModule('core')) from build.gradle files under x-pack
* Removes a place holder test in favor of disabling the test task (in the async plugin)
2020-08-03 13:11:43 -05:00
James Rodewig 26d51089da
[DOCS] Replace `twitter` dataset in docs (#60604) (#60609) 2020-08-03 13:31:19 -04:00
Julie Tibshirani fc63f8224f
Simplify class hierarchy for ordinals field data. (#60606)
This PR simplifies the hierarchy for ordinals field data classes:
* Remove `AbstractIndexFieldData`, since only `AbstractIndexOrdinalsFieldData`
inherits directly from it.
* Make `SortedSetOrdinalsIndexFieldData` extend
`AbstractIndexOrdinalsFieldData`. This lets us remove some redundant code.
2020-08-03 09:58:29 -07:00
James Rodewig c5f4f91ac4 [DOCS] Clarify reindex does not require existing dest 2020-08-03 12:46:40 -04:00
James Rodewig 5be515f126 [DOCS] Unhide EQL search in data streams docs 2020-08-03 11:59:12 -04:00
Hendrik Muhs 1e01832b0c fix possible NPE introduced in #60591 2020-08-03 16:40:38 +02:00
James Rodewig cfab3bccab
[DOCS] Replace `twitter` dataset in cat API docs (#60588) (#60597) 2020-08-03 10:22:36 -04:00
Hendrik Muhs cd6492fc11 [Transform] fix regression of date histogram optimization (#60591)
fixes mix up of input and output field name for date histogram optimization.

minimal fix, more tests to be added with #60469

fixes #60590
2020-08-03 15:52:08 +02:00
James Rodewig bffe82449a
[DOCS] Update Amazon Linux AMI link in EC2 docs (#60589) (#60593) 2020-08-03 09:32:31 -04:00
István Zoltán Szabó 7c995aa3ff [DOCS] Fixes broken links in rest API spec. (#60582) 2020-08-03 15:31:25 +02:00
Yannick Welsch 3409e019d2 Ignore shutdown when retrying recoveries (#60586)
Avoids failures when shutting down a node.
2020-08-03 15:14:38 +02:00
Nik Everett 2cde43b799
Allows nanosecond resolution in search_after (backport of #60328) (#60426)
Allows nanosecond resolution in search_after (#60328)

This fixes `search_after` to properly parse string formatted dates that
have nanosecond resolution.

Closes #52424
2020-08-03 08:17:48 -04:00
Yannick Welsch b0d601fa63 Adjust searchable snapshot license (#60578)
No longer needs Platinum license for testing on staging.
2020-08-03 13:19:53 +02:00
Yannick Welsch 9e24a54382 Clean existing index folder when loading searchable snapshot (#60122)
Closing a regular index and mounting a snapshot-backed index into that existing index does not clean the existing index
folders of those preexisting shards.

This PR removes the existing Lucene / translog files once the searchable snapshot shard is starting up. Future PRs will
make reuse of the existing index files to populate the cache.
2020-08-03 13:19:11 +02:00
David Turner d2ddf8cd6a Improve deserialization failure logging (#60577)
Today when a node fails to properly deserialize a transport message with
a parent task we log the following relatively uninformative message:

    java.lang.IllegalStateException: Message not fully read (response) for requestId [9999], handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.transport.TransportService$6@abcdefgh], error [false]; resetting

In particular, the wrapping of the listener in the `TransportService`
obscures all clues as to the source of the problem, e.g. the action name
or the identity of the underlying listener. This commit exposes the
inner listener to the logs.

Also if the listener is wrapped with `ContextPreservingActionListener`
then its identity is similarly hidden. This commit also exposes the
wrapped listener in this case.

Relates #38939
2020-08-03 11:51:01 +01:00
Yang Wang a76fc324d4
Fix get-license test failure by ensure cluster is ready (#60498) (#60569)
When a new cluster starts, the HTTP layer becomes ready to accept incoming 
requests while the basic license is still being populated in the background. 
When a get license request comes in before the license is ready, it can get 
404 error. This PR fixes it by either wrap the license check in assertBusy or 
ensure the license is ready before perform the check.

This is a backport for both #60498 and #60573
2020-08-03 19:40:03 +10:00
Armin Braun 3270cb3088
More Efficient Writes for Snapshot Shard Generations (#60458) (#60575)
Same as #59905 but for shard level metadata. Since we wnat to retain
the ability to do safe+atomic writes for non-uuid shard generations
this PR has to create two separate write paths for both kinds of
shard generations.
2020-08-03 11:11:36 +02:00
Armin Braun 204efe9387
Add Repository Setting to Disable Writing index.latest (#60448) (#60576)
Writing the `index.latest` blob is unnecessary unless the contents of the repository
are to be used as a URL-repository. Also, in some edge cases, the fact that `index.latest` is the only
blob in the repository that regularly gets overwritten was causing compatibility issues with
some backing blobstores (Azure no-overwrite policy, Hitachy S3 equivalent).
=> this commit changes behavior to make snapshots not fail if writing `index.latest` fails
and adds a setting to disable writing `index.latest`.
2020-08-03 11:11:24 +02:00
Martijn van Groningen e76447833a
adjusted bwcVersions file 2020-08-03 10:58:24 +02:00
Tim Vernum 1a373b0c21
Only call listener once (SP template registration) (#60567)
This fixes a bug in the IdP's template registration that would
sometimes call the listener twice.

Resolves: #54285
Resolves: #54423

Backport of: #60497
2020-08-03 13:45:16 +10:00
Andrei Dan ac258f10d6
Data streams: throw ResourceAlreadyExists exception (#60518) (#60536)
For consistency reasons (and reducing the overload of IllegalArgumentException)
this changes the exception thrown when trying to create a data stream
that already exists.

(cherry picked from commit ac2184c4614bba0f3ee377da49aea0daed98bab4)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-08-01 16:31:09 +01:00
James Rodewig fcc53d9e0e
[DOCS] Note refresh requests are synchronous (#60540) (#60550) 2020-07-31 16:22:34 -04:00
James Rodewig 5a2c6f0d4f
[DOCS] http -> https, remove outdated plugin docs (#60380) (#60545)
Plugin discovery documentation contained information about installing
Elasticsearch 2.0 and installing an oracle JDK, both of which is no
longer valid.

While noticing that the instructions used cleartext HTTP to install
packages, this commit replaces HTTPs links instead of HTTP where possible.

In addition a few community links have been removed, as they do not seem
to exist anymore.

Co-authored-by: Alexander Reelsen <alexander@reelsen.net>
2020-07-31 16:16:31 -04:00
James Rodewig fb599dc343
[DOCS] Add rollups to `Tune for disk usage` (#60436) (#60542)
Co-authored-by: Leaf-Lin <39002973+Leaf-Lin@users.noreply.github.com>
2020-07-31 16:10:57 -04:00
James Rodewig 19c12cc896
[DOCS] Fix merge conflict in `CONTRIBUTING.md` (#60537) 2020-07-31 16:04:18 -04:00
Armin Braun 8c7eae15ba
Increase Timeout in testSnapshotRestore (#60532) (#60538)
It seems this test only fails with `FsRepository` and mostly just barely times out (takes just a little over 30s to go green).
I think just increasing the timeout should be fine as a fix here as it's a little interesting to check larger amounts of
data in this test generally speaking.

Closes #39299
2020-07-31 21:53:58 +02:00
Julie Tibshirani f1d4fd8c3e Correct name of IndexFieldData#loadGlobalDirect. (#60492)
It seems 'localGlobalDirect' was just a typo.
2020-07-31 10:53:21 -07:00
James Rodewig 771e9f142a
[DOCS] Move search pagination content to one page (#60515) (#60525) 2020-07-31 12:40:40 -04:00
James Rodewig 9eba7f39b0
[DOCS] Replace `twitter` dataset in docs APIs (#60521) (#60529) 2020-07-31 12:40:03 -04:00
Julie Tibshirani fe1d877b69 Avoid using string 'y' in fields REST tests. (#60471)
Some yaml parsers interpret 'y' and 'yes' as the boolean 'true'.
2020-07-31 09:12:15 -07:00
Jim Ferenczi 8db896d290 Fix race condition in SearchPhaseControllerTests#testPartialMergeFailure (#60488)
This change ensures that we call the listener for partial merge failure **before**
calling the completion listener in order to avoid race condition in tests.

Closes #60446
2020-07-31 16:29:20 +02:00
James Rodewig 4b12e69e8e
[DOCS] Replace `twitter` dataset in index API docs (#60473) (#60510) 2020-07-31 09:51:47 -04:00
Dan Hermann bd6b20a4ec
Fix failing test for resolve index API (#60306) (#60508) 2020-07-31 08:43:24 -05:00
Rene Groeschke ed4b70190b
Replace immediate task creations by using task avoidance api (#60071) (#60504)
- Replace immediate task creations by using task avoidance api
- One step closer to #56610
- Still many tasks are created during configuration phase. Tackled in separate steps
2020-07-31 13:09:04 +02:00
Hendrik Muhs a721d6d19b [Transform] use correct version in BWC serialization test (#60500)
use correct version in BWC serialization test

fixes #60464
2020-07-31 11:23:05 +02:00
Rene Groeschke 79ec0f2442
Split internal distribution handling into separate internal plugin (7.x backport) (#60270)
* Split internal distribution handling into separate internal plugin (#60295)
* Provide proper failure if unexpected non jdk bundled bwc version is requested
2020-07-31 09:30:21 +02:00
Julie Tibshirani 8ac81a3447 Remove IndexFieldData#clear since it is unused. (#60475)
This method was never called. It also seemed tricky that calling a method on
`IndexFieldData` could clear the contents of a shared cache.
2020-07-30 14:07:55 -07:00
James Rodewig 0022d316bb
[DOCS] Merge search topic and overview pages (#60459) (#60479) 2020-07-30 16:45:18 -04:00
James Rodewig 134b69d3aa
[DOCS] Fix `template` param in put index template API (#60474) (#60476) 2020-07-30 16:44:50 -04:00
Lisa Cawley 9425de8dd1
[DOCS] Clarify support for run as in OIDC realms (#60246) (#60485) 2020-07-30 13:39:41 -07:00
Julie Tibshirani dfd7f226f0
Clarify SourceLookup sharing across fetch subphases. (#60484)
The `SourceLookup` class provides access to the _source for a particular
document, specified through `SourceLookup#setSegmentAndDocument`. Previously
the search context contained a single `SourceLookup` that was shared between
different fetch subphases. It was hard to reason about its state: is
`SourceLookup` set to the expected document? Is the _source already loaded and
available?

Instead of using a global source lookup, the fetch hit context now provides
access to a lookup that is set to load from the hit document.

This refactor closes #31000, since the same `SourceLookup` is no longer shared
between the 'fetch _source phase' and script execution.
2020-07-30 13:22:31 -07:00
Dan Hermann da69644498
Ignore template warnings in IndexingIT (#59962) (#60482) 2020-07-30 14:54:30 -05:00
James Rodewig 2d2b74dd32
[DOCS] Note remote reindex is not fwd compatible (#60425) (#60454) 2020-07-30 09:23:55 -04:00