Commit Graph

5574 Commits

Author SHA1 Message Date
Shay Banon 058f57b198 test for primary allocating over replicas
still marked as await fix, requires investigation
2013-09-03 11:58:14 +02:00
Luca Cavanna 2d1b841ba1 Replaced localhost with 127.0.0.1 in PluginManagerTests 2013-09-03 11:44:56 +02:00
Luca Cavanna 45c8da3e98 Improved logic that removes top-level folder from archives when needed
Whether we remove the top-level folder from the archive depends now on the zip itself and not on where it was downloaded from. That makes it work installing local files too.

Closes #3582
2013-09-03 10:34:32 +02:00
Simon Willnauer eb2fed85f1 Add 'min_input_len' to completion suggester
Restrict the size of the input length to a reasonable size otherwise very
long strings can cause StackOverflowExceptions deep down in lucene land.
Yet, this is simply a saftly limit set to `50` UTF-16 codepoints by default.
This limit is only present at index time and not at query time. If prefix
completions > 50 UTF-16 codepoints are expected / desired this limit should be raised.
Critical string sizes are beyone the 1k UTF-16 Codepoints limit.

Closes #3596
2013-09-03 10:26:37 +02:00
Martijn van Groningen eed7f0bdb3 Restructured the transport action code for multi percolate api. 2013-09-03 10:19:58 +02:00
Boaz Leskes 7307e37efe Previously added logging in TransportRefreshAction caused NPE if search shards had errors (or was not yet started/assigned)
Better formatting for logging in TransportRefreshAction
2013-09-02 21:53:00 +02:00
Boaz Leskes a48c437ffb Search action now log in debug what shards/nodes were used before moving to second phase
Added  a debug log for executing a refresh request on a shard.
2013-09-02 17:17:53 +02:00
Luca Cavanna 91c3853452 Added new iteration to RecoveryWhileUnderLoadTests
We want to check whether the issue causing the test to fail would be fixed by an additional refresh or not
2013-09-02 16:29:49 +02:00
Luca Cavanna fc1d6670df Added log lines to log (debug level) acknowledgements regarding index alias creation and index state change 2013-09-02 12:16:14 +02:00
Luca Cavanna 38be478c46 Fixed close index when the index is already closed
Closes #3601
2013-09-02 10:58:09 +02:00
Luca Cavanna 206ab3a6d4 Used ElasticsearchAssertions to check count result in RecoveryWhileUnderLoadTests 2013-09-02 10:30:10 +02:00
Clinton Gormley ca4b85edef Added IDs to the community clients docs, to control HTML page names 2013-09-01 12:58:17 +02:00
Simon Willnauer 69848210d6 Allow up to 10% unbalance in random allocator tests 2013-08-31 09:13:54 +02:00
Boaz Leskes be09103258 SearchWhileCreatingIndexTests didn't always supply enough node for write consistency requirements and thus hanged.
Added more debug logging to TransportShardReplicationOperationAction

*Changed* node naming in tests from GUIDs to node# based naming as they are much easier to read
2013-08-30 20:56:23 +02:00
Martijn van Groningen be00437c65 Changed expectedOperationsPerItem from AtomicInteger array to AtomicReferenceArray<AtomicInteger>.
Made node stopping / starting less aggressive in RecoveryPercolatorTests.
2013-08-30 16:10:53 +02:00
Simon Willnauer c9c850167f Add more trace logging if shard allocation is throttled 2013-08-30 14:05:53 +02:00
Boaz Leskes c70b8fbd7b Added total, successful shard info to search related asserts in ElasticsearchAssertions
Rewrote SimpleSearchTests assertion to use that.
2013-08-30 14:02:13 +02:00
Simon Willnauer f8cf5ae7e9 Added Random test for ShardAllocator.
This test will make random allocation decision on a growing and
shrinking cluster leading to a random distribution of the shards.
After a certain amount of iterations the test allows allocation unless
the same shard is already allocated on a node and balances the cluster
to gain optimal balance.
2013-08-30 10:11:45 +02:00
Martijn van Groningen a7b2b7847a Use atomic collections to make sure all of the memory contents are visible from writing to reading thread. 2013-08-30 00:06:45 +02:00
Simon Willnauer 7113731022 Execute listeners on current thread if threadpool is shutting down 2013-08-29 16:40:38 +02:00
Simon Willnauer f408673bd4 Force default codec in tests where we rely on the latest lucene defaults 2013-08-29 16:40:38 +02:00
Nik Everett 10e55bd3ef Recheck cutoffScore during phrase_suggest merge.
The goal is to throw out suggestions that only meet the cutoff in some
shards.  This will happen if your input phrase is only contained in a
few shards.  If your shards are unbanced this rechecking can throw out
good suggestions.

Closes #3547.
2013-08-29 16:40:38 +02:00
Martijn van Groningen 76939b82d3 Removed unnecessary catch clause.
Improved logging
2013-08-29 16:00:59 +02:00
Boaz Leskes ffd019c07e Logging shard level multi percolate errors under debug rather than trace 2013-08-29 14:15:38 +02:00
Luca Cavanna 649108656d Added logging and asserts to make it easier to understand potential failures 2013-08-29 14:10:49 +02:00
Simon Willnauer 8b617fb48f Convert relative paths to URI first in tests to prevent URL encoded paths
If a path to the test classes contains spaces Hunspell tests failed due
to URL encoded paths. This happens on CI builds if you give the build
a name containing a space. This is fixed by first converting to a URI
and created a File object from the URI directly.
2013-08-29 12:56:33 +02:00
Martijn van Groningen 25c1b93d57 Ignore AlreadyExpiredException exception during when indexing a document on a replica.
The document should just be indexed to stay consistent with the primary shard.
2013-08-29 11:38:13 +02:00
Martijn van Groningen a1f2f44eef Make sure 2 writes happened, so we can check if 2 deletes happen that have been triggered by ttl 2013-08-29 11:27:01 +02:00
Adrien Grand 5b6be0c456 Use a separate build directory for Eclipse.
The fact that Maven and Eclipse share the same build directories can trigger
race conditions when both are trying to build at the same time, eg. if you run
`mvn clean test` while Eclipse is up and running: Eclipse will notice that some
class files are missing and start compiling in parallel with Maven.
2013-08-29 10:29:26 +02:00
Simon Willnauer 11ff90420a Fixed typo 2013-08-29 10:27:20 +02:00
Boaz Leskes e807c99f27 Fixed a typo in the config of light finnish stemmer (old last_finish is still supported for backward compatibility)
Closes #3594
2013-08-29 10:15:40 +02:00
Clinton Gormley 822043347e Migrated documentation into the main repo 2013-08-29 01:24:34 +02:00
Simon Willnauer b9558edeff Prevent allocation algorithm from prematurely exiting balance operations
In a special case if allocations from the "heaviest" to the "lighter" nodes is not possible
due to some allocation decider restrictions like zone awareness. if one zone has for instance
less nodes than another zone so one zone is horribly overloaded from a balanced perspective but we
can't move to the "lighter" shards since otherwise the zone would go over capacity.

Closes #3580
2013-08-28 23:31:35 +02:00
Simon Willnauer 82397c0554 Treat empty prefrence as a `not set` in Plain Operation Routing
An empty preference was causing a AIOOB exception in
PlainOperationRouting. We now check for `null` or `empty` instead of
just `null`

Closes #3591
2013-08-28 20:57:39 +02:00
Boaz Leskes 86cb76a0ce Added a status method CountResponse that resolve shard failures into rest status code. That method is now used in RestCountAction to return proper status.
Closes #3585
2013-08-28 20:21:22 +02:00
Shay Banon db11c30dd5 batch failed shards into a single cluster state event
make sure we process as much as possible failed shard events within a single cluster state event callback (similar to what we do with started shards)
2013-08-28 15:29:16 +02:00
Adrien Grand b63af53313 Make BytesValues documentation clearer about BytesRef ownership. 2013-08-28 15:24:51 +02:00
Martijn van Groningen e6f014bd62 no need for flush/optimize 2013-08-28 14:56:26 +02:00
Shay Banon 25d42e5caf Don't recover a shard if it just failed on this node and wasn't reassigned to this node by master yet.
additional places where we should track failed shards, and call clean as part of the top level calls
2013-08-28 14:12:07 +02:00
Igor Motov ed2740a50a Don't recover a shard if it just failed on this node and wasn't reassigned to this node by master yet.
When recovery of a shard fails on a node, the node sends notification to the master with information about the failure. During the period between the shard failure and the time when notification about the failure reaches the master, any changes in shard allocations can trigger the node with the failed shard to allocate this shard again. This allocation (especially if successful) creates a ripple effect of the shard going through failure/started states in order to match the delayed states processed by master.  Under certain condition, a node involved in this process might generate warning messages: "marked shard as started, but shard has not been created, mark shard as failed".

This fix makes sure that nodes keep track of failed shard allocations and will not try to allocate such shards repeatedly while waiting for the failure notification to be processed by master.
2013-08-28 14:12:07 +02:00
Britta Weber 513c761aee ElasticsearchAssertions.assertHitCount also checks shard failures 2013-08-28 10:47:39 +02:00
Martijn van Groningen 60ac34ff3a Added `_name` support to queries.
This extends the named filter support from only filters to also queries.

Closes #3581
2013-08-28 10:42:53 +02:00
Boaz Leskes 45d4864021 Refreshing after green so also recovering replicas will be refreshed. 2013-08-28 08:44:10 +02:00
Simon Willnauer e6dcd137a6 AwarenessAllocationTests must extend ESTestCase in order to respect AwaitsFix annotations 2013-08-27 19:28:20 +02:00
Lee Hinman 9d5868904b Recover small files (< 1mb) using a separate threadpool than large files.
Fixes #3576
2013-08-27 11:01:54 -06:00
Britta Weber 1ab037d4d0 Fix searching while shard is being relocated
Shard relocation caused queries to fail in the fetch phase either with a
`AlreadyClosedException` or a `SearchContextMissingException`.
This was caused by the `SearchService` releasing the `SearchContext`s via the
superfluous call of `releaseContextsForShard()` when the
2013-08-27 18:30:00 +02:00
Simon Willnauer e7ff8ea509 Added unit tests or zone awareness 2013-08-27 18:05:42 +02:00
Luca Cavanna d5b2c8e82f Fixed extraction of site plugins downloaded from github, so that we skip the top-level folder and we place the files directly under the _site folder
Closes #3551
2013-08-27 17:03:55 +02:00
Boaz Leskes fa067e058a MultiTermVectorsAction was wrongly registered under mget 2013-08-27 13:46:33 +02:00
Simon Willnauer 39040b5f17 Add more tests for Zone Awareness 2013-08-27 12:25:35 +02:00