Commit Graph

11228 Commits

Author SHA1 Message Date
Michael McCandless 0683a66277 Core: remove index.fail_on_merge_failure
Always fail the engine if an unexpected exception is hit during merge.

Closes #10088
2015-03-14 09:53:42 -04:00
javanna ebe1a4663d [TEST] renamed variables in ScriptServiceTests 2015-03-13 12:59:40 -07:00
Lee Hinman 6aec68cd29 Revert "[QUERY] Remove lowercase_expanded_terms and locale options"
This reverts commit d1f7bd97cb.

Ryan pointed out that this needs to work with the multi term query, so
additional analysis and tests should be added.
2015-03-13 13:51:44 -06:00
Lee Hinman d1f7bd97cb [QUERY] Remove lowercase_expanded_terms and locale options
The analysis chain should be used instead of relying on this, as it is
confusing when dealing with different per-field analysers.

The `locale` option was only used for `lowercase_expanded_terms`, which,
once removed, is no longer needed, so it was removed as well.

Fixes #9978
Relates to #9973
2015-03-13 13:17:27 -06:00
mikemccand ee8aaead78 Core: don't rethrow merge exceptions
We already 1) log the full exception, and 2) fail the engine.

Closes #10083
2015-03-13 10:59:01 -07:00
Boaz Leskes 3fbd934e30 Translog: make sure stats's op count and size are in sync
During translog flush there is a very small window where translog stats reports can be inconsistent.

Closes #10041
2015-03-13 10:42:58 -07:00
javanna a8271595dc Scripting: File scripts cache key to include language to prevent conflicts
The file scripts cache key should include the language of the script to prevent conflicts between scripts with same name but different extension (hence lang). Note that script engines can register multiple acronyms that may be used as lang at execution time (e.g. javascript and js are synonyms). We then need to make sure that the same script gets loaded no matter which of the acronyms is used at execution time. The problem didn't exist before this change ad the lang was simply ignored, while now we take it into account.

This change has also some positive effect on inline scripts caching. Up until now, the same script referred to with different acronyms would be compiled and cached multiple times, once per acronym. After this change every script gets compiled and cached only once, as we chose internally the acronym used as part of the cache key, no matter which one the user provides.

Closes #10033
2015-03-12 23:18:00 -07:00
Petr Bela f27cb07eb9 [DOCS] fix typo in scripting docs 2015-03-12 15:28:50 -07:00
Simon Willnauer 8b4f6e0111 [SHARD] make assert less strict to ensure local node is not null 2015-03-12 12:11:20 -07:00
Igor Motov 55f2a547d0 Snapshot/Restore: delete operation should ignore finalizing shards on nodes that no longer exist
Related to #9924
2015-03-11 12:10:48 -07:00
olivier bourgain 2c4aaa0838 [DOCS] fix typo in BoolQueryBuilder javadocs 2015-03-10 08:21:34 -07:00
olivier bourgain bcb4decca9 [DOCS] add missing comma in percentile_rank aggregation example 2015-03-10 08:21:06 -07:00
olivier bourgain fb7cd2ea9a [DOCS] Adjusted geo_distance aggregation example
unit is not returned in the response, but we have key and an implicit from starting at 0 for the first bucket
2015-03-10 08:20:20 -07:00
olivier bourgain eaeddc6bd4 [DOCS] missing curly brace in ip_range aggregation example 2015-03-10 08:19:57 -07:00
David Pilato d9c19cd846 [doc] Cat API: show open and closed indices in _cat/indices
Related to #7936
2015-03-09 15:45:26 -07:00
Simon Willnauer 20f98fe34a Improve reporting of assert in IndexShard 2015-03-09 11:12:45 -07:00
Lee Hinman 0fdc425591 Fix extra logging parameter in RecoverySourceHandler 2015-03-08 17:56:00 -06:00
Shay Banon a161cf402b fix test to close node environment
the resource check only fails on windows for some reason..., need to chase why...
2015-03-08 16:42:18 -07:00
Shay Banon 13630d8028 Remove dangling indices settings, always import it
Remove the settings around dangling indices, such as no import and timeout for deletion, we always want to import dangling indices for safety, and we should not allow to change the behavior. This also cleans up the code quite a bit.
closes #10016
2015-03-08 13:24:07 -07:00
Boaz Leskes 67df205b23 Gateway: improve assertion at the end of shard recovery
we want to make sure the recovery finished all the way to post recovery. Current check, validating the shard is either in POST_RECOVERY or STARTED is not good because the shard could be also closed if things go fast enough (like in our tests). The assertion is changed to check the shard is not left in CREATED or RECOVERING.

Closes #10028
2015-03-08 10:55:02 -07:00
javanna 521ac7f35a Scripting: cleanup ScriptService & friends in preparation for #6418
- Added NAME constants for each script language, avoiding to repeat the same strings all over the place.
- Simplified `compile` method signatures by removing a couple of variants. Note that all of these signatures are going to change again with #6418 as in order to compile/execute a script the caller will need to specify which operation is attempting to execute the script, info that will be provided as an additional mandatory argument.
- Removed double call to ScriptService#verifyDynamicScripting for every indexed or dynamic script.
- Decreased ScriptService inner classes visibility to private (CacheKey, IndexedScript, ApplySettings)
- Moved ScriptService inner classes to the bottom of the class, I think it makes it more readable.
- Resolved some compiler warnings

Closes #9992
2015-03-07 10:01:34 +01:00
javanna 2d2ba48c0b Tribe node: system properties and configuration settings must not be forwarded to tribe clients
The tribe node, at startup, sets up the tribe clients that will join their corresponding tribes. All of the tribe.* settings are properly forwarded to the corresponding tribe client. System properties and global configuration settings must not be forwarded to the tribe client though or they will end up overriding per tribe settings with same name causing issues.

 For instance if you set the transport.tcp.port to some defined value for the tribe node, via system property or configuration file, that same value must not be forwarded to the tribe clients, otherwise they will try and use the same port, which will be already occupied by the tribe node itself, resulting in startup failed. Same for cluster.name, which will cause the tribe clients not to join their tribes.

Closes #9576
Closes #9721
2015-03-07 09:26:46 +01:00
Matthias Wahl e15d4d1124 Internal: Fix equality check of timevalue after serialization
closes #9218
2015-03-06 13:00:31 -08:00
Lee Hinman 583c4926c4 Mark shadow replicas with 's' in _cat/shards output
Fixes #9772
2015-03-06 13:21:29 -07:00
Britta Weber 0ca1e31392 Revert "[mappings] update dynamic fields in mapping on master even if parsing fails for the rest of doc"
This reverts commit d9a1540948.
2015-03-06 19:23:41 +01:00
Britta Weber 3abf11611b [significant terms] Use 'script_file' instead of 'script' if the script is a file 2015-03-06 18:28:33 +01:00
Britta Weber 580728dfd6 significant terms: add scriptable significance heuristic
This commit adds scripting capability to significant_terms.
Custom heuristics can be implemented with a script that provides
parameters subset_freq, superset_freq,subset_size, superset_size.

closes #7850
2015-03-06 17:06:04 +01:00
Britta Weber d9a1540948 [mappings] update dynamic fields in mapping on master even if parsing fails for the rest of doc
The local DocumentMapper is updated while parsing and dynamic fields are added before
parsing has finished. If parsing fails after a dynamic field has been added already
then the field was not added to the cluster state but was present in the local mapper of this
node. New documents with the same field would not necessarily cause an update either and
after restarting the node the mapping for these fields were lost. Instead the new fields
should always be updated.

closes #9851
closes #9874
2015-03-06 15:35:23 +01:00
Martijn van Groningen dccaa49aa0 Tests: wait for discovery to be completed after both majority nodes follow a new master 2015-03-06 13:50:34 +01:00
Shay Banon 48bdd58d51 Use provided cluster state for indices service validations
Since the method can be called in an #execute event of the cluster service, we need the ability to use the cluster state that will be provided in the ClusterChangedEvent, have the ClusterState be provided as a parameter
2015-03-06 13:39:53 +01:00
Shay Banon 0a3175a16f Fix Java 8 _ variable warning
The _ variable causes a warning when compiling with Java 8, noting that it might be removed in a future version
2015-03-06 12:41:53 +01:00
Simon Willnauer c5a6767a55 [TEST] Reduce the number of fields in MLT tests 2015-03-06 11:54:27 +01:00
Martijn van Groningen 634c91a0ca Tests: added more logging 2015-03-06 11:18:24 +01:00
Britta Weber 6c79636985 [TEST] wait until mappings are added to cluster state to make sure they are persisted 2015-03-06 10:42:29 +01:00
Martijn van Groningen 0c254e9832 Zen: Node receiving a cluster state with a wrong master node should reject and throw an error.
Previously it was ignored and the publish cluster state timeout would kick in. In that case a stale master node would just wait for the inevitable and waste valuable time.
This issue was discovered by the DiscoveryWithServiceDisruptionsTests#testStaleMasterNotHijackingMajority test.

Also only perform cluster state versions and wrong master node check inside cluster state update task.
2015-03-06 08:47:32 +01:00
Lee Hinman 2d2cc764ac Revert "[TESTS] remove AwaitsFix from CircuitBreakerServiceTests"
This reverts commit 21e246f6a3.
2015-03-05 22:13:05 -07:00
Lee Hinman 91ce4830b7 [ENGINE] Inc store reference before reading segments info
If a tragic even happens while we are reading the segments info from the
store the store might have been closed concurrently. We had this
behavior before and was lost in a refactoring.
2015-03-05 14:16:21 -07:00
Simon Willnauer 565ee05cda [ENGINE] Inc store reference before reading segments info
If a tragic even happens while we are reading the segments info
from the store the store might have been closed concurrently. We had this behavior
before and was lost in a refactoring.
2015-03-05 21:41:01 +01:00
Martijn van Groningen 274da68040 Test: added health call to will hold when wait when there is a cluster block, so that the refresh call doesn't fail if that is the case. 2015-03-05 20:32:23 +01:00
Britta Weber cea8999406 Store: Delete index folder if all shards were allocated away from a data only node
If a folder for an index was created that folder is never deleted from that node unless the index is deleted.
Data only nodes therefore can have empty folders for indices that they do not even have shards for.
This commit makes sure empty folders are cleaned up after all shards have moved away from a data only
node. The behavior is unchanged for master eligible nodes.

closes #9985
2015-03-05 15:48:41 +01:00
Matias Tealdi cba6dff3ac fixing typo in expDecayFunction and adding offset to all dacay functions
closes #9887
2015-03-05 12:28:08 +01:00
Colin Goodheart-Smithe 5c44db50bc [DOCS] update JAVA API with aggregation changes
The Histogram and Range APIs for the aggregations changed so that there was a common interface between he types of Range/Histogram. This PR reflects that change in the Java API docs

Contributes to #9976
2015-03-05 11:09:49 +00:00
Colin Goodheart-Smithe 8935959cbf [TEST] removed custom index settings for all DateHistogram tests
In favour of randomising number of shards and replicas and only the extended bounds test sets number of shards and replicas explicitly
2015-03-05 10:29:22 +00:00
Michael McCandless 24d8c74a52 Core: throttle delete-by-query when merges are falling behind
Delete-by-query is incredibly costly because it forces a refresh each
time, so if you are also indexing this can cause massive segment
explosion.

This change throttles delete-by-query when merges can't keep up.  It's
likely not enough (#7052 is the long-term solution) but can only
help.

Closes #9986
2015-03-04 15:25:10 -05:00
Britta Weber 764901a9cd [TEST] make sure update task is actually executed
The update task that was submitted in this test never got executed
if node_2 was not master.
2015-03-04 15:49:49 +01:00
Simon Willnauer 6b95a05fc7 Revert "[TEST] Enable trace logging for translog recovery"
This reverts commit 52d53e658f.
2015-03-04 15:33:28 +01:00
Martijn van Groningen f81805d7d3 Test: muted testStaleMasterNotHijackingMajority 2015-03-04 15:07:59 +01:00
Colin Goodheart-Smithe 010493682c Search: fixed log to print throwable correctly
In TransportSearchTypeAction one of the logger calls was passing the throwable in as a parameter for the message rather than a throwable to be printed as a stack trace. This change fixes it so the throwable is printed properly
2015-03-04 13:50:23 +00:00
Simon Willnauer 52d53e658f [TEST] Enable trace logging for translog recovery 2015-03-04 11:48:10 +01:00
Lee Hinman a5ab49dc70 [TESTS] Flush and refresh before querying in IndexWithShadowReplicasTests 2015-03-03 14:35:22 -07:00