Commit Graph

19603 Commits

Author SHA1 Message Date
Jason Tedor cd56366378 Assert that we fail the correct shard when a replication request fails
This commit adds an assertion to
TransportReplicationActionTests#runReplicateTest that when a replication
request fails, we fail the correct shard.
2016-01-06 11:01:02 -05:00
Martijn van Groningen 2d6adf6428 Percolator refactoring:
* Added percolator field mapper that extracts the query terms and indexes these terms with the percolator query.
* At percolate time these extracted terms are used to query percolator queries that are like to be evaluated. This can significantly cut down the time it takes to percolate. Whereas before all percolator queries were evaluated if they matches with the document being percolated.
* Changes made to percolator queries are no longer immediately visible, a refresh needs to happen before the changes are visible.
* By default the percolate api only returns upto 10 matches instead of returning all matching percolator queries.
* Made percolate more modular, so that it is easier to add unit tests.
* Added unit tests for the percolator.

Closes #12664
Closes #13646
2016-01-06 16:08:10 +01:00
Nik Everett 646d24ebdc Add a test that the typename isn't highlighted 2016-01-06 09:20:15 -05:00
Yannick Welsch e0599bfb66 Merge pull request #15785 from ywelsch/fix/path-hierarchy-type
Add PathHierarchy type back to path_hierarchy tokenizer for backward compatibility with 1.x
2016-01-06 14:38:14 +01:00
Yannick Welsch de6dfe15a7 Add PathHierarchy type back to path_hierarchy tokenizer for backward compatibility with 1.x
Closes #15785
2016-01-06 14:37:33 +01:00
Yannick Welsch a6ec1434d6 [TEST] Reduce log level in NodeVersionAllocationDeciderTests 2016-01-06 14:35:47 +01:00
Simon Willnauer 8a90c8085d Merge pull request #15788 from s1monw/dont_delete_tlog_file
Never delete translog-N.tlog file when creation fails
2016-01-06 14:31:22 +01:00
Simon Willnauer 5c833750d7 apply feedback from @bleskes 2016-01-06 14:19:58 +01:00
Luca Cavanna eabd5e2714 Merge pull request #15787 from javanna/enhancement/processor_package
move all the processors under the same package org.elasticsearch.ingest.processor
2016-01-06 13:28:35 +01:00
Simon Willnauer 2f3de1719f Merge pull request #15786 from s1monw/dont_call_listener_under_lock
Never call a listerner under lock in InternalEngine
2016-01-06 13:23:31 +01:00
Simon Willnauer 12b93e72f0 Never delete translog-N.tlog file when creation fails
We today delete the translog-N.tlog file if any subsequent operation fails
but we might actually be in a good state if for instance the creation of the writer
failes after we sucessfully baked the new translog generation into the checkpoint. In this situation
we used to delete the translog-N.tlog file and failed on the next recovery of the translog with a
NoSuchFileException | FileNotFoundException just like in https://discuss.elastic.co/t/cannot-recover-index-because-of-missing-tanslog-files/38336

This commit changes the behavior and cleans up that limbo state on recovery if we already have a generation+1 file written but not baked into
the checkpoint we remove that file but only if the previous ckp file has already been renamed otherwise we know we can't be in this state.
2016-01-06 13:10:21 +01:00
javanna 1e8995d984 move all the processors under the same package org.elasticsearch.ingest.processor 2016-01-06 12:36:28 +01:00
Simon Willnauer 56329d0f53 Never call a listerner under lock in InternalEngine
We has a postIndex|DeleteUnderLock listener callback to load percolator
queries which is entirely private to the index shard in the meanwhile. Yet,
it still calls an external callback while holding an indexing lock which is scary
since we have no control over how long the operation could possibly take.

This commit decouples the percolator registry entirely from the ShardIndexingService
by pessimistically fetching percolator documents from the the engine using realtime get.
Even in situations where the same document is changed concurrently we will eventually end up
in the correct state without loosing an update. This also moves the index throtteling stats directly into
the engine to entirely remove the need for the dependency between InternalEngine and ShardIndexingService.
2016-01-06 11:38:34 +01:00
Imran Azad 8081c782ef Documented search_quote_analyzer in mapping types and detailed how to disable stop words as a potential use case. 2016-01-06 10:40:51 +01:00
Colin Goodheart-Smithe 2c33f78192 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/children/ChildrenParser.java
#	core/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParser.java
#	test/framework/src/main/java/org/elasticsearch/test/TestSearchContext.java
2016-01-06 09:35:53 +00:00
Yannick Welsch 501497df1c Merge pull request #15775 from ywelsch/fix/node-versions-compatiblity
Prevent peer recovery from node with older version
2016-01-06 10:08:01 +01:00
Yannick Welsch 55cc88e1ae Fix version-based allocation decider to prevent peer recovery from node with older version
Relocating a non-primary shard from one node to another is actually done by recovering from the active
primary shard in the cluster, and not the node that we are logically relocating from.

Closes #15775
2016-01-06 10:07:39 +01:00
Michael McCandless 3744fb9dc0 merge master 2016-01-06 04:03:42 -05:00
Michael McCandless 5f4afe842c remove nocommits; fix test case 2016-01-06 03:41:23 -05:00
Adrien Grand 7e3ccf2ee3 Merge pull request #15746 from jpountz/fix/missing_terms_agg
Make `missing` on terms aggs work with all execution modes.
2016-01-06 09:32:39 +01:00
Jason Tedor d032dabed5 Merge pull request #15777 from jasontedor/safer-cluster-state-task-notifications
Safe cluster state task notifications
2016-01-05 16:56:24 -05:00
Jason Tedor 05c46c9d35 Safe cluster state task notifications
This commit addresses an issue where a cluster state task listener
throwing an exception could prevent other listeners from being notified,
and could prevent the executor from receiving notifications that a new
cluster state was published. Additionally, this commit also addresses a
similar issue for executors handling cluster state publication
notifications.
2016-01-05 16:44:59 -05:00
Igor Motov 9ca4386fba Lower logging level for registering/unregistering tasks with the task manager
At the debug level the task manager is too talkative.
2016-01-05 15:15:02 -05:00
Tal Levy 11d4417251 [doc] update set and append processor doc examples 2016-01-05 11:45:52 -08:00
Tal Levy 15ecf76d54 [doc] update convert processor example format 2016-01-05 09:54:13 -08:00
Igor Motov a89dba27c2 Task Management: Add framework for registering and communicating with tasks
Adds task manager class and enables all activities to register with the task manager. Currently, the immutable Transport*Activity class represents activity itself shared across all requests. This PR adds and an additional structure Task that keeps track of currently running requests and can be used to communicate with these requests using TransportTaskAction.

Related to #15117
2016-01-05 12:24:43 -05:00
Simon Willnauer dff30ece05 Double check if stream must be flush to allow tests to make better assumptions of what is visible and what isn't after tragic events 2016-01-05 17:58:20 +01:00
Michael McCandless 1d46a00d43 move async-ness upwards 2016-01-05 11:27:05 -05:00
Michael McCandless 8e7719d136 add logger.warn if thread pool size is clipped; fix test failure 2016-01-05 10:59:40 -05:00
Simon Willnauer ea6718d878 Add pending review from @bleskes on #15771 2016-01-05 16:34:26 +01:00
Simon Willnauer cb2e9c5a43 Merge pull request #15771 from s1monw/simplify_translog_writer
Simplify TranslogWriter to always write to a stream
2016-01-05 16:28:39 +01:00
Tal Levy d1356b0a6f Merge pull request #15763 from talevy/fix/simulate
[Ingest] Fix/simulate
2016-01-05 07:21:36 -08:00
Jason Tedor 5226186a81 Merge pull request #15772 from jasontedor/test-cluster-service-no-timeout
Fix NPE in TestClusterService when waiting indefinitely
2016-01-05 09:57:06 -05:00
Michael McCandless 99d6ec53fa fold in feedback 2016-01-05 09:53:13 -05:00
Jason Tedor 419f3976c2 Fix NPE in TestClusterService when waiting indefinitely
When waiting indefinitely for a new cluster state in a test,
TestClusterService#add will throw a NullPointerException if the timeout
is null. Instead, TestClusterService#add should guard against a null
timeout and not even attempt to add a notification for the timeout
expiring. Note that the usage of null is the agreed upon contract for
specifying an indefinite wait from ClusterStateObserver.
2016-01-05 09:52:29 -05:00
Simon Willnauer 5272c98136 format finally block 2016-01-05 15:50:10 +01:00
Simon Willnauer 469a179ab1 remove stale comments 2016-01-05 15:48:35 +01:00
Simon Willnauer 2370ace922 assert that tragic even exception is never null 2016-01-05 15:45:41 +01:00
Simon Willnauer f69502dd04 Simplify TranslogWriter to always write to a stream
We used to write into an in-memory buffer and if necessary also allow reading
from the memory buffer if the some translog locations that are not flushed to
the channel need to be read. This commit hides all writing behind a buffered output
stream and if ncecessary flushes all buffered data to the channel for reading. This allows
for several simplifcations like reusing javas build in BufferedOutputStream and removes the
need for read write locks on the translog writer. All thread safety is now achived using
the synchronized primitive.
2016-01-05 15:22:56 +01:00
Eugene Pirogov d48af9a155 Fix indent in example
Previously it would look like if `warnings` key is nested under `errors`.
2016-01-05 14:41:09 +01:00
Adrien Grand 2c87010487 Merge pull request #15722 from mysqlguru/master
DOC - field name is changed
2016-01-05 14:34:21 +01:00
Adrien Grand 12e1b7e456 Merge pull request #15740 from PowerToFly/master
fix syntax in the request params
2016-01-05 14:29:44 +01:00
Adrien Grand 2fb4be5064 Merge pull request #15739 from omiend/omiend-patch-1
add double quotation
2016-01-05 14:21:45 +01:00
Simon Willnauer 107859f347 Merge pull request #15766 from s1monw/add_more_io_exceptions
Add serialization support for more important IOExceptions
2016-01-05 12:00:35 +01:00
Simon Willnauer 3a41dfe1dd Add serialization support for more important IOExceptions
Several IOExceptions are always wrapped in an NotSerializableWrapper which is
annoying to read. These exceptions are important to get right across the network
and we should support the important ones that indicate problems on the Filesystem.

This commit also adds general support for IOException to preserve the parent type
across the network if no specific type is serializable.
2016-01-05 10:30:26 +01:00
Adrien Grand 1fcf9958b2 Fix GeoPointFieldMapperTests expectations. 2016-01-05 10:26:39 +01:00
Simon Willnauer bf7e2c333a Merge pull request #14632 from s1monw/indexing_stats_javadocs_
Add javadocs to IndexingStats.Stats
2016-01-05 09:37:42 +01:00
Simon Willnauer ea2a8150d3 Merge pull request #15692 from s1monw/issues/15675
Fix pharse suggest collate example to use correct script context/syntax
2016-01-05 09:27:18 +01:00
Simon Willnauer d4de8dbcfe Merge pull request #15762 from s1monw/issues/15754
Close recovered translog readers if createWriter fails
2016-01-05 08:23:11 +01:00
Tal Levy 183386173c cleanup simulate test and add docs 2016-01-04 16:10:42 -08:00