Commit Graph

8896 Commits

Author SHA1 Message Date
Simon Willnauer 3900d61254 [TESTS] Add ElasticsearchSingleNodeTest.
This test makes it easy to create a lightweight node (no http, indices stored
in RAM, ...) whose main purpose is to get an instance of the Guice injector
for unit tests.

This should help not have to update lots of unit tests when we add a new
Guice dependency.
2014-07-15 15:42:58 +02:00
Patrick Peschlow 9742d08f53 [UPDATE] Handle document missing exception on retry correctly.
Throwables thrown on update retries are now caught and handled via
the provided callback. This commit also contains an integration test
demonstrating the bug and validating the fix.

Closes #6355
Closes #6724
2014-07-15 14:52:29 +02:00
javanna 3c54eb9b8f Internal: make transport action name available in TransportAction base class
Each transport action is associated with at least an action name, which is the action name that gets serialized together with the request and identifies what to do with the request itself. Also, the action name is the name of the registered transport handler that handles incoming request for the transport action.

This commit makes the action name available in a generic manner in the TransportAction base class, so that it can be used when needed by subclasses, or in the base class for instance for action filtering.

Closes #6860
2014-07-15 14:35:35 +02:00
mikemccand 4194ab31c8 Core: don't close/reopen IndexWriter when changing RAM buffer size
Today we close/reopen IW when we change the RAM buffer but that's
costly because it means the next NRT reader is a full reopen.  The RAM
buffer size setting is a live one in IndexWriter, even if there are no
buffered docs in RAM when you call it.

Separately it would be nice if Lucene let you manage a "reader pool"
that could outlive individual IW instances ...

Closes #6856
2014-07-15 08:32:30 -04:00
Simon Willnauer 557b634f4a [SEARCH] Wrap filter only once in ApplyAcceptedDocsFilter
We potentially wrap the given filter multiple times when iterating the
subreaders. We only need to do this once.

Closes #6873
2014-07-15 12:44:35 +02:00
Simon Willnauer 7de9d3d2cb [TEST] Fix test to use keyword analyzer since it relies on exact terms without processing 2014-07-15 12:26:51 +02:00
Simon Willnauer 82cc227da3 Added missing version based serialization 2014-07-15 12:13:45 +02:00
Brian Murphy 73d93e5fd8 Indexed Scripts/Templates: cleanup
This commit cleans up some code around the indexed script/templates feature.
Remove dead code in ScriptService.
Remove setXScript methods for UpdateRequestBuilder and use setScript(script,type) instead
2014-07-15 10:49:39 +01:00
Simon Willnauer 2bac455cd0 Add version check before reading script type in UpdateRequest 2014-07-15 10:16:08 +02:00
Simon Willnauer 70bd24ed14 [TEST] use higher timeout to wait for balanced cluster
CorruptFileTest sometimes hits conditions where lots of rebalancing
happens. In such a case the default timeout is just not enough - this
timeout just makes sure that the cluster has enough time to balance
itself.
2014-07-15 09:56:41 +02:00
Areek Zillur 0bd774423d [TEST] fix random preference string tests to accomodate for the new more strict preference type 2014-07-14 20:15:00 -04:00
Areek Zillur d0d1b98d23 Stats: Expose IndexWriter and VersionMap RAM usage to ShardStats and _cat endpoint
This commit adds the RAM usage of IndexWriter and VersionMap

Closes #6483
2014-07-14 19:46:12 -04:00
Areek Zillur 4fb79fe787 [TEST] fix random preference string test to accomodate for the new more strict preference type 2014-07-14 19:01:04 -04:00
Areek Zillur 0173298f29 [TEST] fix PhraseInIndex test bug for shard failure 2014-07-14 17:12:51 -04:00
Areek Zillur 76343899ea Phrase Suggester: Add collate option to PhraseSuggester
The newly added collate option will let the user provide a template query/filter which will be executed for every phrase suggestions generated to ensure that the suggestion matches at least one document for the filter/query.
The user can also add routing preference `preference` to route the collate query/filter and additional `params` to inject into the collate template.

Closes #3482
2014-07-14 16:07:52 -04:00
Simon Willnauer f8be82f54a [TEST] close IW before listing directory content 2014-07-14 22:00:03 +02:00
uboness 04b412b597 Added more utility methods to Settings
- names() to return the direct settings names
- getAsSettings(String) to return the settings mapped to the given name (like getByPrefix(...) except no need to provide a tailing '.')
2014-07-14 20:25:35 +02:00
mikemccand f392a99bd9 Test: add diagnostics to get some details next time this test fails 2014-07-14 13:48:56 -04:00
Brian Murphy f68a06ff99 [TEST] Fix test relying on order of json for matching 2014-07-14 17:06:23 +01:00
Brian Murphy 9c0b25dcce [TEST] Fix test issues with branches still using mvel as default scripting language 2014-07-14 16:51:17 +01:00
Simon Willnauer c91ec7c730 [TEST] fix compile problems on Java 8 2014-07-14 17:17:24 +02:00
Malte Schirnacher 647a2a64a1 Docs: Update query-string-syntax.asciidoc
Closes #6853
2014-07-14 16:35:17 +02:00
Brian Murphy e79b7086de Indexed Scripts/Templates: Add support for storing/deleting/getting scripts/template from an index.
This change allow elasticsearch users to store scripts and templates in an index for use at search time.
Scripts/Templates are stored in the .scripts index. The type of the events is set to the script language.
Templates use the mustache language so their type is be "mustache".
Adds the concept of a script type to calls to the ScriptService types are INDEXED,INLINE,FILE.
If a script type of INDEXED is supplied the script will be attempted to be loaded from the indexed, FILE will
look in the file cache and INLINE will treat the supplied script argument as the literal script.
REST endpoints are provided to do CRUD operations as is a java client library.
All query dsl points have been upgraded to allow passing in of explicit script ids and script file names.
Backwards compatible behavior has been preserved so this shouldn't break any existing querys that expect to
pass in a filename as the script/template name. The ScriptService will check the disk cache before parsing the
script.

Closes #5921 #5637 #5484
2014-07-14 14:37:55 +01:00
Boaz Leskes dcb2107a1d [Test] testSingleNodeWithFlush should wait for yellow
O.w. it can restart a node before all primaries are started , leading to a red state down the road...
2014-07-14 15:13:52 +02:00
mikemccand 80774877ff Test: add test case verifying updating merge IO throttle settings works
Closes #6842
2014-07-14 08:37:29 -04:00
Martijn van Groningen b275393e01 [TEST] For connection rules make TransportAddress the identity instead of DiscoveryNode.
For unicast ping the DiscoveryNode identity is based on its id, which in that stage is a dummy value, this breaks any rule in the mock tran
However the TransportAddress is a valid value in unicast ping and all other places, so that is a better alternative.

Closes #6836
2014-07-14 12:52:45 +02:00
Clinton Gormley 6e70edb0a4 Analysis: Improve Hunspell error messages
The Hunspell service would throw a confusing error message if more than
one affix file was present.  This commit distinguishes between the two
error cases: where there are no affix files and when there are too many
affix files.

Also implements lazy dictionary loading, which was used in the tests
but not implemented.

Closes #6850
2014-07-14 12:13:32 +02:00
Britta Weber 74927adced significant terms: infrastructure for changing easily the significance heuristic
This commit adds the infrastructure to allow pluging in different
measures for computing the significance of a term.
Significance measures can be provided externally by overriding

- SignificanceHeuristic
- SignificanceHeuristicBuilder
- SignificanceHeuristicParser

closes #6561
2014-07-14 11:00:50 +02:00
Boaz Leskes 8865e60e93 [Transport] possible NPE during shutdown for requests using timeouts
Closes #6849
2014-07-14 10:52:29 +02:00
Simon Willnauer 86bc79202d [ENGINE] Mark store as corrupted before sending failed shard
We have to mark a shard as corrupted if necessary before the
shard failed event is fired ie. before we call the corresponding
listener in the engine. Otherwise the shard might be re-allocated
on the same node and just started up without being marked as corrupted.

Relates to #5924
2014-07-14 10:14:58 +02:00
Simon Willnauer e8ff007852 [RECOVERY] Increment Store refcount on RecoveryTarget
We should make sure we have incremented the store refcount
before we start the recovery on the recovyer target.

Closes #6844
2014-07-14 09:18:25 +02:00
Boaz Leskes ab11c6821d [Test] one more tweak to testLocalNodeMasterListenerCallbacks 2014-07-13 17:59:45 +02:00
Boaz Leskes c3e842e363 [Test] renamed testListenerCallbacks to testLocalNodeMasterListenerCallbacks
Also clean up internal variable namings and fixed usage of wrong setting causing last node to not use the min_master_node settings.
2014-07-13 17:51:01 +02:00
Martijn van Groningen af38b9f7ba Core: Added missing return statements.
Closes #6841
2014-07-13 15:53:05 +02:00
Igor Motov 60b317caa4 Snapshot/Restore: Add ability to restore indices without their aliases
Closes #6457
2014-07-13 17:52:41 +09:00
Shay Banon f7a88fdd3e [TEST] wait for green before deleting mapping 2014-07-13 17:21:26 +09:00
Shay Banon fb6d847aac [TEST] wait for green before deleting mapping 2014-07-13 17:17:14 +09:00
Boaz Leskes 5e3742762a [Test] testHostOnMessages - only decrease latch after setting transport addresses 2014-07-12 09:11:27 +02:00
Florian Hopf 3689f67a76 Docs: Fixed invalid word count in geodistance agg doc
Closes #6838
2014-07-11 18:35:36 +02:00
Martijn van Groningen 05ca763b10 [TEST] Ensure that one node is part of the cluster. 2014-07-11 17:51:35 +02:00
Martijn van Groningen 6547ff3eb0 Print trace log if not enough master nodes could be found. 2014-07-11 17:42:11 +02:00
uboness 25a21c6a01 Cleanup of the transport request/response messages
Now both TransportRequest and TransportResponse inherit from a base TransportMessage that holds the message headers and also now added the remote transport address (where this message came from).
2014-07-11 16:41:01 +02:00
Boaz Leskes c4c0270c52 [Tests] Enhance ZenUnicastDiscoveryTest
This started out as a simple correction to a missing setting problem, but go bigger into more general work on the ZenUnicastDiscoveryTets suite. It now works with both network and local mode. I also merge the different ZenUnicast test suites into a single place.

Closes #6835
2014-07-11 16:37:52 +02:00
mikemccand 6c78147f5f Docs: remove orphan comma 2014-07-11 08:26:08 -04:00
Britta Weber 6d8fff65dc Throw exception if function in function score query is null
closes #6292 #6784
2014-07-11 13:57:11 +02:00
mikemccand b4e80999a7 Docs: fix merge docs to match the code (the max_thread_count default is 'aggressive' (favor SSDs)) 2014-07-11 07:00:57 -04:00
Shay Banon 43a5cbe9be Only use IndexShard instance to lookup recovery status
make sure we use the instance itself to look it up, and not the shard id, as we might get another instance
leftover from #6825
2014-07-11 11:38:36 +02:00
Boaz Leskes f480969503 [Gateway] set a default of 5m to `recover_after_time` when any to the `expected*Nodes` is set
The `recovery_after_time` tells the gateway to wait before starting recovery from disk. The goal here is to allow for more nodes to join the cluster and thus not start potentially unneeded replications. The `expectedNodes` setting (and friends) tells the gateway when it can start recovering even if the `recover_after_time` has not yet elapsed. However, `expectedNodes` is useless if one doesn't set `recovery_after_time`. This commit changes that by setting a sensible default of 5m for `recover_after_time` *if* a `expectedNodes` setting is present.

Closes #6742
2014-07-11 11:28:45 +02:00
Alex Ksikes af4eee594c More Like This: ensures selection of best terms is indeed O(n)
Previously the size of the priority queue was wrongly set to the total number
of terms. Instead, it should be set to 'maxQueryTerms'. This makes the
selection of best terms O(n), instead of O(n*log(n)).

Jira patch: https://issues.apache.org/jira/browse/LUCENE-5795

Closes #6657
2014-07-11 11:14:31 +02:00
Shay Banon 01ca81e2a3 Improve handling of failed primary replica handling
Out of #6808, we improved the handling of a primary failing to make sure replicas that are initializing are properly failed as well. After double checking it, it has 2 problems, the first, if the same shard routing is failed again, there is no protection that we don't apply the failure (which we do in failed shard cases), and the other was that we already tried to handle it (wrongly) in the elect primary method.
This change fixes the handling to work correctly in the elect primary method, and adds unit tests to verify the behavior
The change also expose a problem in our handling of replica shards that stay initializing during primary failure and electing another replica shard as primary, where we need to cancel its ongoing recovery to make sure it re-starts from the new elected primary
closes #6825
2014-07-11 10:51:59 +02:00