Commit Graph

28030 Commits

Author SHA1 Message Date
Martijn van Groningen a2b4080fba
use diamond operator 2017-06-29 13:43:39 +02:00
Christoph Büscher aa2038f9d7 Use DocumentField#toXContent and parsing in SearchHit (#25469)
As a small follow-up to #25361, we can use DocumentFields toXContent/fromXContent
in SearchHit now.
2017-06-29 13:32:13 +02:00
olcbean 3518e313b8 Unify the result interfaces from get and search in Java client (#25361)
As GetField and SearchHitField have the same members, they have been unified into
DocumentField.

Closes #16440
2017-06-29 11:35:28 +02:00
Jason Tedor da59c178e2 Emit settings deprecation logging at most once
When a setting is deprecated, if that setting is used repeatedly we
currently emit a deprecation warning every time the setting is used. In
cases like hitting settings endpoints over and over against a node with
a lot of deprecated settings, this can lead to excessive deprecation
warnings which can crush a node. This commit ensures that a given
setting only sees deprecation logging at most once.

Relates #25457
2017-06-28 22:18:46 -04:00
DeDe Morton 9714c77c84 [Docs] Add link to grok debugger docs (#25461) 2017-06-28 16:28:15 -07:00
DeDe Morton 6442d1f75e [Docs] Add link to grok debugger docs (#25412) 2017-06-28 16:14:16 -07:00
Ali Beyad b18bfd6062 Output all empty snapshot info fields if in verbose mode (#25455)
In #24477, a less verbose option was added to retrieve snapshot info via
GET /_snapshot/{repo}/{snapshots}.  The point of adding this less
verbose option was so that if the repository is a cloud based one, and
there are many snapshots for which the snapshot info needed to be
retrieved, then each snapshot would require reading a separate snapshot
metadata file to pull out the necessary information.  This can be costly
(performance and cost) on cloud based repositories, so a less verbose
option was added that only retrieves very basic information about each
snapshot that is all available in the index-N blob - requiring only one
read!

In order to display this less verbose snapshot info appropriately, logic
was added to not display those fields which could not be populated.
However, this broke integrators (e.g. ECE) that required these fields to
be present, even if empty.  This commit is to return these fields in the
response, even if empty, if the verbose option is set.
2017-06-28 17:37:56 -05:00
Jay Modi 64d11b8831 Fix race condition in RemoteClusterConnection node supplier (#25432)
This commit fixes a race condition in the node supplier used by the RemoteClusterConnection. The
node supplier stores an iterator over a set backed by a ConcurrentHashMap, but the get operation
of the supplier uses multiple methods of the iterator and is suceptible to a race between the
calls to hasNext() and next(). The test in this commit fails under the old implementation with a
NoSuchElementException. This commit adds a wrapper object over a set and a iterator, with all methods
being synchronized to avoid races. Modifications to the set result in the iterator being set to null
and the next retrieval creates a new iterator.
2017-06-28 15:50:24 -06:00
Tim Brooks cad57959e1 Remove finicky exception message assertion
In SimpleNioTransportTests we assert that an IOException has a certain
message. This message appears that it is not dependible (and might
change based on platform).

Our other transport tests (mock and netty) do not make this assertion.
Instead they only assert on our application exception message. This
commit removes the IOException message assertion. And retains the
ConnectTransportException message assertion.
2017-06-28 14:16:04 -05:00
Jay Modi b2901f536e Do not search locally if remote index pattern resolves to no indices (#25436)
This commit changes how we determine if there were any remote indices that a search should have
been executed against. Previously, we used the list of remote shard iterators but if the remote
index pattern resolved to no indices there would be no remote shard iterators even though the
request specified remote indices. The map of remote cluster names to the original indices is used
instead so that we can determine if there were remote indices even when there are no remote shard
iterators.

Closes #25426
2017-06-28 12:41:37 -06:00
Andreas Gebhardt a156ccd80e Expand `/_cat/nodes` to return information about hard drive (#21775)
Expand `/_cat/nodes` with already present information about available disk space `diskAvail` (alias: `d`, `disk`) by:

    * `diskTotal` (alias `dt`): total disk space
    * `diskUsed` (alias `du`): used disk space (`diskTotal - diskAvail`)
    * `diskUsedPercent` (alias `dup`): used disk space percentage

Note: The available disk space is the number of bytes available to the node's Java virtual machine. The size might be smaller than the real one. That means the used disk space (percentage) is larger.

Closes #21679
2017-06-28 18:20:20 +02:00
Tim Brooks 5f8be0e090 Introduce NioTransport into framework for testing (#24262)
This commit introduces a nio based tcp transport into framework for
testing.

Currently Elasticsearch uses a simple blocking tcp transport for
testing purposes (MockTcpTransport). This diverges from production
where our current transport (netty) is non-blocking.

The point of this commit is to introduce a testing variant that more
closely matches the behavior of production instances.
2017-06-28 10:51:20 -05:00
Martijn van Groningen 9ce9c21b83
docs: added percolator script query limitation 2017-06-28 17:10:30 +02:00
Chris Earle f2eeceb10d _nodes/stats should not fail due to concurrent AlreadyClosedException (#25016)
This catches `AlreadyClosedException` during `stats` calls to avoid failing a `_nodes/stats` request because of the ignorable, concurrent index closure.
2017-06-28 10:08:45 -04:00
Yannick Welsch 5a4a47332c Use a single method to update shard state
This commit refactors index shard to provide a single method for
updating the shard state on an incoming cluster state update.

Relates #25431
2017-06-28 09:48:47 -04:00
Jason Tedor ebdae09df3 Do not swallow exception when relocating
When relocating a shard before changing the state to relocated, we
verify that a relocation is a still taking place. Yet, this can throw an
exception if the relocation is in fact no longer valid. Sadly, we were
swallowing the exception in this situation. This commit allows such an
exception to bubble up after safely releasing resources.
2017-06-28 08:42:13 -04:00
Robin Clarke 1900d9c447 Docs: Fix typo for request cache (#25444) 2017-06-28 14:31:03 +02:00
Jason Tedor be906628d5 Remove implicit 32-bit support
We previously tried to maintain (while not formally supporting) 32-bit
support, although we never tested this anywhere in CI. Since we do not
formally support this, and 32-bit usage is very low, we have elected to
no longer maintain 32-bit support. This commit removes any implication
of 32-bit support.

Relates #25435
2017-06-28 08:24:33 -04:00
Marcus Wittig 960d63a3b3
[DOCS] reworded to prevent code span rendering glitch (#25442)
Changed `rescore`s to `rescore` requests as an backtick followed by the s character appears to be interpreted as an apostrophe  which then leads to an unbalanced backtick for the next code span in the remainder of the paragraph

Closes #25443
2017-06-28 11:48:31 +01:00
Yannick Welsch 5d1e67c882 Disallow multiple concurrent recovery attempts for same target shard (#25428)
The primary shard uses the GlobalCheckPointTracker to track local checkpoint information of recovering and started replicas in order to calculate the global checkpoint. As the tracker is updated through recoveries as well, it is easier to reason about the tracker if we can ensure that there are no concurrent recovery attempts for the same target shard (which can happen in case of network disconnects).
2017-06-28 10:41:16 +02:00
Yannick Welsch 8ae61c0fc4 Update global checkpoint when increasing primary term on replica (#25422)
When a replica shard increases its primary term under the mandate of a new primary, it should also update its global checkpoint; this gives us the guarantee that its global checkpoint is at least as high as the new primary and gives a starting point for the primary/replica resync.

Relates to #25355, #10708
2017-06-28 10:38:22 +02:00
Daniel Mitterdorfer dd6751d3e9 Add backwards compatibility indices for 5.4.3 2017-06-28 10:00:01 +02:00
Daniel Mitterdorfer 75ceb7d63b Add version 5.4.3 after release 2017-06-28 09:59:54 +02:00
Russ Cam 2804fc4c29 Update MSI installer images (#25414)
Slight updating to styling
2017-06-28 09:04:23 +10:00
Jason Tedor 8afeeed051 Add missing newline at end of SetsTests.java
This commit adds a missing newline to the end of SetsTests.java after
the closing curly brace.
2017-06-27 17:28:41 -04:00
Jason Tedor f6a693e1bc Rename handoff primary context transport handler
This commit renames this handler from "hand_off" to "handoff" since
"handoff" is an actual word in the English language.
2017-06-27 15:08:58 -04:00
Tal Levy cbcf6a4f55 correct expected thrown exception in mappingMetaData to ElasticsearchParseException (#25410) 2017-06-27 08:55:24 -07:00
Martijn van Groningen c85ac402b0
test: Make many percolator integration tests real integration tests 2017-06-27 17:44:30 +02:00
Lisa Cawley 03f952a838 [DOCS] Update docs to use shared attribute file (#25403)
* [DOCS] Update docs to use shared attribute file

* [DOCS] Add shared attributes to Versions.asciidoc
2017-06-27 08:33:28 -07:00
Jason Tedor 9b3768204b Add Javadocs and tests for set difference methods
This commit adds Javadocs and tests for some set difference utility
methods in core.
2017-06-27 11:29:35 -04:00
Christoph Büscher c55dc23270 Tests: Add parsing test for AggregationsTests (#25396)
We already have these tests in InternalAggregationTestCase to check random insertions into the response xContent so that we don't fail on future changes in the response format. This change adds the same to AggregationsTests and runs on a whole aggregations tree. Unfortunately we need to exclude many places in the xContent from random insertion, but I added a long comment trying to explaine those.
2017-06-27 17:02:15 +02:00
Martijn van Groningen 11fcfaae68
test: get upgrade status for all indices
Relates to #25311
2017-06-27 16:56:48 +02:00
Daniel Mitterdorfer 0405ef5892 Mute SignificantTermsAggregatorTests#testSignificance()
Relates #25429
2017-06-27 15:58:22 +02:00
Martijn van Groningen 9f5aef7b6d
test: added extra log line
Relates to #25311
2017-06-27 10:54:37 +02:00
Daniel Mitterdorfer 54907ba352 Mute FullRollingRestartIT#testFullRollingRestart()
Relates #25420
2017-06-27 10:41:48 +02:00
Daniel Mitterdorfer ef9d099ffd Mute IndexShardTests#testRelocatedShardCanNotBeRevivedConcurrently 2017-06-27 10:25:40 +02:00
Jason Tedor f27aba34bf Mark shutdown non-master nodes test as awaits fix
This commit marks a failing test as awaits fix. The test is failing due
to a primary shard not knowing its own local checkpoint in the global
checkpoint tracker after recovery. If such a shard becomes primary after
promotion, and is then subsequently relocated, it can lead to a
violation of an assertion that when the primary context is transferred
the knowledge of all in-sync local checkpoints is consistent with the
global checkpoint on the relocation target.

Relates #25415
2017-06-26 22:48:04 -04:00
Jason Tedor dfd241e0a6 Remove default path settings
This commit removes the default path settings for data and logs. With
this change, we now ship the packages with these settings set in the
elasticsearch.yml configuration file rather than going through the
default.path.data and default.path.logs dance that we went through in
the past.

Relates #25408
2017-06-26 21:43:20 -04:00
Jason Tedor cca18a2c35 Make plugin loading stricter
Today we load plugins reflectively, looking for constructors that
conform to specific signatures. This commit tightens the reflective
operations here, not allowing plugins to have ambiguous constructors.

Relates #25405
2017-06-26 21:42:53 -04:00
Deb Adair 2765ea41ca [DOCS] Fixed broken cross doc links to security settings. 2017-06-26 17:48:56 -07:00
Jason Tedor 22beb8d03c Remove hacky node version check in NodeInfo
This commit removes a hacky way of checking that a node is running
Elasticsearch 5.x when starting standalone nodes in tests.

Relates #25406
2017-06-26 20:32:13 -04:00
Ryan Ernst 5de406debb Fix docs lucene version check error message 2017-06-26 15:45:13 -07:00
Ryan Ernst 5726d1394f Build: Add check on lucene version in docs (#25407)
This commit adds a version verifica†ion to ensure the lucene version in
the docs is up to date.
2017-06-26 14:22:54 -07:00
Jason Tedor 5a9fc8aa2a Remove path.conf setting
This commit removes path.conf as a valid setting and replaces it with a
command-line flag for specifying a non-default path for configuration.

Relates #25392
2017-06-26 15:18:29 -04:00
Alexander Kazakov 53b74348ff Fix documentation for script processor (#25299) 2017-06-26 12:14:23 -07:00
Jason Tedor e9e7007a51 Remove LongTuple
This commit removes an abstraction that was introduced when introducing
the primary context. As this abstraction is used in exactly one place,
we simply make that abstraction local to its usage so that we do not
accumulate yet another general abstraction with exactly one usage.

Relates #25402
2017-06-26 14:46:06 -04:00
Jason Tedor 56d3a5e6d8 Fix primary context sealing test
This commit updates some assertions in the primary context sealing test
after the restriction on updating allocation IDs from master and
updating global checkpoint on replica while sealed were removed.
2017-06-26 14:17:33 -04:00
Jason Tedor c6a03bc549 Introduce primary context (#25122)
* Introduce primary context

The target of a primary relocation is not aware of the state of the
replication group. In particular, it is not tracking in-sync and
initializing shards and their checkpoints. This means that after the
target shard is started, its knowledge of the replication group could
differ from that of the relocation source. In particular, this differing
view can lead to it computing a global checkpoint that moves backwards
after it becomes aware of the state of the entire replication
group. This commit addresses this issue by transferring a primary
context during relocation handoff.

* Fix test

* Add assertion messages

* Javadocs

* Barrier between marking a shard in sync and relocating

* Fix misplaced call

* Paranoia

* Better latch countdown

* Catch any exception

* Fix comment

* Fix wait for cluster state relocation test

* Update knowledge via upate local checkpoint API

* toString

* Visibility

* Refactor permit

* Push down

* Imports

* Docs

* Fix compilation

* Remove assertion

* Fix compilation

* Remove context wrapper

* Move PrimaryContext to new package

* Piping for cluster state version

This commit adds piping for the cluster state version to the global
checkpoint tracker. We do not use it yet.

* Remove unused import

* Implement versioning in tracker

* Fix test

* Unneeded public

* Imports

* Promote on our own

* Add tests

* Import

* Newline

* Update comment

* Serialization

* Assertion message

* Update stale comment

* Remove newline

* Less verbose

* Remove redundant assertion

* Tracking -> in-sync

* Assertions

* Just say no

Friends do not let friends block the cluster state update thread on
network operations.

* Extra newline

* Add allocation ID to assertion

* Rename method

* Another rename

* Introduce sealing

* Sealing tests

* One more assertion

* Fix imports

* Safer sealing

* Remove check

* Remove another sealed check
2017-06-26 14:09:15 -04:00
Nik Everett 4306315ff6 Throw useful error on bad docs snippets (#25389)
You can continue a test started in a previous snippet by marking the
next snippet with `// TEST[continued]`. The trouble is, if you mark the
first snippet in a file with `// TEST[continued]` you'd get difficult
to predict behavior because you'd continue the test started in another
file. This will usually create a test that fails the build but it isn't
easy to track down what you did wrong. This commit catches this
scenario up front and fails the build with a useful error message.

Similarly, if you put `// TEST[continued]` directly after a
`// TESTSETUP` section then the docs tests will fail to run but the
error message did not point you to the `// TEST[continued]` snippet.
This commit catches this scenario up front as well and fails the build
with a useful error message.
2017-06-26 11:52:16 -04:00
Simon Willnauer d338a09812 Remove `mapping.single_type` from parent join test (#25391)
This removes the remaining usage of `mapping.single_type` from the parent join
module and moves it's bwc test to the mixed cluster tests

Relates to #24961
Relates to #20257
2017-06-26 17:33:07 +02:00