Commit Graph

18772 Commits

Author SHA1 Message Date
Adrien Grand 055953d6b3 Expose the reason why a mapping merge is issued.
This would be useful in order to only perform some validations in the case of
a mapping update and in cases when a mapping is restored eg. after a restart,
such as discussed in #15989.

This replaces the current `applyDefault` parameter which can be derived from
the mapping merge reason: the default mapping should be applied only in case of
a mapping update, if the mapping does not exist yet and if this is not the
default mapping.
2016-01-18 17:41:23 +01:00
Clinton Gormley e5034c9c1f Merge pull request #15973 from DynamicScope/patch-1
Update bulk.asciidoc
2016-01-18 12:00:02 +01:00
Alexander Reelsen 2b0c036b99 Tests: Ensure termvector is not fetched from translog
Setting realTime to false in the get term vector request ensures,
that after a refresh the document is not fetched from the translog,
which seems to yield different in rare test runs.

The change likely triggering this was introduced in #15933
2016-01-18 10:18:43 +01:00
Jason Tedor 69b21feb3b Merge pull request #15748 from jasontedor/shard-failure-no-master-retry
Wait for new master when failing shard

Relates #14252
2016-01-17 10:50:58 -05:00
Jason Tedor 386d2ab86a More tightening of shard state action tests
This commit adds some additional assertions that test success is not
falsely indicated by adding assertions that success / failure methods
are not incorrectly invoked in failure / success scenarios.
2016-01-17 10:36:24 -05:00
Jason Tedor 3d329e3ba0 Add real clock test for age of PrioritizedRunnable
This commit adds a test for calculating the age of PrioritizedRunnable
that allows real clock time to elapse. The test ensures that at least
one millisecond has passed, and that the resolution of System#nanoTime
on the underlying system is actually able to detect this.

Relates #15995
2016-01-16 19:01:25 -05:00
Jason Tedor cf3c0ed049 Sanity assertion that exception cause is not null
This commit adds a sanity assertion that the cause of a transport
exception when sending a shard failure is not null.
2016-01-15 16:52:04 -05:00
Jason Tedor b4538a5676 Avoid periodic object allocations in IndexService
This commit wraps a trace logging message in a trace logging level check
to prevent allocating an Object array (to hold the logging parameters)
and a String (from the interval) when trace logging is not enabled every
second (with the default index refresh interval) and every five seconds
(with the default translog sync interval) for every open index when
trace logging is not enabled.
2016-01-15 15:14:09 -05:00
Chris Earle 1926fe5ad5 Merge branch 'master' of https://github.com/elastic/elasticsearch 2016-01-15 14:29:07 -05:00
Chris Earle 8068ceacaa Revert "[DOCS] Clarify Shadow replica setting"
This reverts commit e17b86f0c1.
2016-01-15 14:28:44 -05:00
Jason Tedor a7185a1d31 Simplify equality test in IndexShard#sameException
This commit simplifies an equality test in IndexShard#sameException
where the messages for two exceptions are being compared. The previous
condition first tested logical equality if the left exception is not
null, and otherwise tested reference equality. There is a convenience
method since JDK 7 for testing equality in this way: Objects#equals.

Closes #16025
2016-01-15 14:19:47 -05:00
Britta Weber abc8c02bed check that busy waiting does not time out and fix replica counter test 2016-01-15 18:46:18 +01:00
Jason Tedor 7eefcbbeed Add retry loop in shard state action tests
This commit enhances the master channel exception test in
o.e.c.a.s.ShardStateActionTests to test that a retries loop as expected
when requests to the master repeatedly fail.
2016-01-15 11:26:32 -05:00
Nik Everett 7745c64a62 Merge pull request #15894 from nik9000/fix_blended_terms_2
Fix blended terms take 2
2016-01-15 09:54:58 -05:00
Nik Everett 50098bfb2c Fix blended terms for non-strings take 2
It had some funky errors, like lenient:true not working and queries with
two integer fields blowing up if there was no analyzer defined on the
query. This throws a bunch more tests at it and rejiggers how non-strings
are handled so they don't wander off into scary QueryBuilder-land unless
they have a nice strong analyzer to protect them.
2016-01-15 09:53:31 -05:00
Jason Tedor fe39d11c55 Logging in shard state action
This commit adds a trace log on a cluster state update while waiting for
a new master, and changes the log level on cluster service close to the
warn level.
2016-01-15 09:45:39 -05:00
Jason Tedor efb142613f Tighten shard state action tests
This commit tightens the tests in o.e.c.a.s.ShardStateActionTests:
 - adds a simple test for a success condition that validates the shard
   failed request is correct and sent to the correct place
 - remove redundant assertions from the no master and master left tests
 - an assertion that success is not falsely indicated in the case of a
   unhandled error
2016-01-15 09:28:25 -05:00
Jason Tedor aa28133865 Merge pull request #15995 from jasontedor/prioritized-runnable-age
Fix calculation of age of pending tasks

Closes #15988
2016-01-15 08:01:50 -05:00
Jason Tedor ba8ad9c2b7 Fix calculation of age of pending tasks
This commit addresses a time unit conversion bug in calculating the age
of a PrioritizedRunnable. The issue was an incorrect conversion from
nanoseconds to milliseconds as instead the conversion was to
microseconds. This leads to the timeInQueue metric for pending tasks to
be off by three orders of magnitude.
2016-01-15 07:58:50 -05:00
Adrien Grand 0806190d1a Merge pull request #15998 from jpountz/fix/filter_agg_creates_meights_once
Filter(s) aggregation should create weights only once.
2016-01-15 10:25:51 +01:00
Adrien Grand cc41e6e7fe Filter(s) aggregation should create weights only once.
We have a performance bug that if a filter aggregation is below a terms
aggregation that has a cardinality of 1000, we will call Query.createWeight
1000 times as well. However, Query.createWeight can be a costly operation.
For instance in the case of a TermQuery it will seek the term in every
segment. Instead, we should create the Weight once, and then get as many
iterators as we need from this Weight.

I found this problem while trying to diagnose a performance regression while
upgrading from 1.7 to 2.1[1]. While the problem was not introduced in 2.x, the
fact that 1.7 cached very aggressively had hidden this problem, since you don't
need to seek the term anymore on a cached TermFilter.

Doing things once for every aggregator is not easy with the current API but
I discussed this with Colin and Aggregator factories will need to get an init
method for different reasons, where we will be able to put these steps that
need to be performed only once, no matter haw many aggregators need to be
created.

[1] https://discuss.elastic.co/t/aggregations-in-2-1-0-much-slower-than-1-6-0/38056/26
2016-01-15 09:42:30 +01:00
Martijn van Groningen a05ea535ad percolator: Make sure that start time is serialized on the mpercolate shard requests
Closes #15908
2016-01-15 09:25:01 +01:00
Daniel Mitterdorfer 871b38afcb Check cluster health in integration test wait condition
With this commit we do not check only if an endpoint is up but
we also check that the cluster status is green. Previously,
builds sporadically failed to pass this condition.
2016-01-15 09:14:32 +01:00
Jason Tedor 7f78d52c72 Use capture and clear convenience method 2016-01-14 16:51:36 -05:00
Lee Hinman 03087cfc47 Skip capturing least/most FS info for an FS with no total
If an operating system reports -1 for the total bytes of a filesystem
path, we should ignore it when capturing the least and most available
statistics.

Relates to #15919

Squashed commit of the following:

commit 5d2258ffeff8a0d156295dcc754ab9b6cbb4b02e
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Jan 14 14:14:27 2016 -0700

    Change test to test positive total with negative 'free' value

commit 927e61d4b39692fc147220a955b63b291ad80db5
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Jan 14 13:09:28 2016 -0700

    Skip capturing least/most FS info for an FS with no total

    If an operating system reports -1 for the total bytes of a filesystem
    path, we should ignore it when capturing the least and most available
    statistics.

    Relates to #15919
2016-01-14 14:44:46 -07:00
Chris Earle da316605ac Merge pull request #16005 from pickypg/docs/shadow-replicas-every-data-node-16004
[DOCS] Clarify Shadow replica setting
2016-01-14 16:38:00 -05:00
Chris Earle e17b86f0c1 [DOCS] Clarify Shadow replica setting
Clarifying that the path setting is required on every _data_ node,
rather than _every_ node.
2016-01-14 16:31:35 -05:00
Simon Willnauer 903d0ff96c Use trace for annoying schedule logs 2016-01-14 22:09:45 +01:00
Jason Tedor 8f67dcc348 Add Javadocs for exceptions that are handled by ShardStateAction 2016-01-14 15:07:17 -05:00
Jason Tedor 5a5d7881ed Remove dead field in o.e.c.a.s.ShardStateActionTests 2016-01-14 15:07:17 -05:00
Jason Tedor d55c5f6c4d Handle FailedToCommitClusterStateException in ShardStateAction
This commit adds Discovery.FailedToCommitClusterStateException to the
list of channel failures that ShardStateAction handles and retries.
2016-01-14 15:07:17 -05:00
Jason Tedor f17f9a5f36 Remove timeout mechanism from ShardStateAction
This commit removes the timeout retry mechanism from ShardStateAction
allowing it to instead be handled by the general master channel retry
mechanism. The idea is that if there is a network issue, the master will
miss a ping timeout causing the channel to be closed which will expose
itself via a NodeDisconnectedException. At this point, we can just wait
for a new master and retry, as with any other master channel exception.
2016-01-14 15:07:16 -05:00
Jason Tedor f49435c78b Centrally handle channel failures when failing a shard
This commit moves the handling of channel failures when failing a shard
to o.e.c.a.s.ShardStateAction. This means that shard failure requests
that timeout or occur when there is no master or the master leaves after
the request is sent will now be retried from here. The listener for a
shard failed request will now only be notified upon successful
completion of the shard failed request, or when a catastrophic
non-channel failure occurs.
2016-01-14 15:07:16 -05:00
Jason Tedor 58c2a3b687 Add simulation of master leaving in TransportReplicationActionTests
This commit adds a simulation of the master leaving after a shard
failure request has been sent. In this case, after a new cluster state
is published (simulating a new master having been elected), the request
to fail the shard should be retried.
2016-01-14 15:05:46 -05:00
Jason Tedor 703ff2b5ab Wait for new master when failing shard
This commit handles the situation when we are failing a shard and either
no master is known, or the known master left while failing the shard. We
handle this situation by waiting for a new master to be reelected, and
then sending the shard failed request to the new master.
2016-01-14 15:05:46 -05:00
Kiyan 120210d024 Fix file descriptor endpoint in configuration docs
The endpoint /_nodes/process does not show file descriptors, should be
/_nodes/stats/process.

Closes #16000
2016-01-14 14:33:22 -05:00
Jason Tedor a954e4e8e5 Safely notify cluster state task executor of publish 2016-01-14 10:07:08 -05:00
Christoph Büscher 3d98756e64 Merge branch 'refactor/rescore-builder-equals-hash' 2016-01-14 15:40:34 +01:00
Christoph Büscher 76192024a8 Make RescoreBuilder and nested QueryRescorer Writable
Adding serialization capabilities to RescoreBuilder and make
all QueryRescorer implement NamedWritable, also requiring
all implementations of RescoreBuilder.Rescorer to implement
equals() and hashCode.

In addition, the current rescore mode enumeration is pulled out to a
separate class to make sharing of constants easier between
the query builders XContent rendering coder and the parser.
2016-01-14 15:26:18 +01:00
Luca Cavanna 1cb1607c29 Merge pull request #15954 from javanna/enhancement/get_alias_expand_closed
make get alias expand to open and closed indices by default
2016-01-14 14:24:40 +01:00
Simon Willnauer 4bef475985 [TEST] Add test to show how to parse an enum from a string in ObjectParser.java 2016-01-14 14:20:33 +01:00
Clinton Gormley 9a89a33643 Merge pull request #15984 from pdudits/patch-1
Fix incorrect file-based example in script_fields
2016-01-14 14:17:31 +01:00
David Pilato 5c9bc37450 Merge branch 'pr/azure-repo-no-secondary' 2016-01-14 14:06:30 +01:00
David Pilato ed45ad6327 Fix Azure repository with only one primary account
Using a single azure account is now rejected.
This commit fixes this issue and adds a test for it.

This regression was introduced with #13779. Hopefully no elasticsearch version has been released since then.

Needs to be merged in 2.2, 2.x and master branches.
2016-01-14 13:50:02 +01:00
Simon Willnauer 01989eb426 Merge pull request #15977 from s1monw/trash_flush_on_close_setting
Remove index.flush_on_close entirely
2016-01-14 12:54:05 +01:00
javanna 07a82d0c09 make get alias expand to open and closed indices by default
This change affects get alias, get aliases as well as cat aliases. They all return closed indices too by default. get alias and get aliases also allow to return open indices only through the `expand_wildcards` option (set it to `open`).

Closes #14982
2016-01-14 10:40:31 +01:00
Simon Willnauer aaf7e55c5f Remove index.flush_on_close entirely
This undocumented setting was mainly used for testing and a safety net for
a new flushOnClose feature back in 1.x. We can now safely remove this setting.
The test usage now uses a mock plugin setting to achive the same.
2016-01-14 10:34:58 +01:00
Daniel Mitterdorfer dc51dd0056 Add more logging to wait condition for elasticsearch nodes in tests 2016-01-14 09:48:27 +01:00
Clinton Gormley b1ee074921 Document that bool.filter assigns scores of 1.0 2016-01-14 09:28:32 +01:00
Jason Tedor d5e20fb763 Merge pull request #15934 from jasontedor/freebsd-load-average
Add load averages to OS stats on FreeBSD

Closes #15917
2016-01-13 20:11:22 -05:00