Commit Graph

6069 Commits

Author SHA1 Message Date
Simon Willnauer 012d47b500 Use debug logging rather than info for rejected ping task
This exception is thrown on node shutdown and doesn't indicate
an critical situation but rather is caught for consistency reasons.
2013-08-02 14:10:55 +02:00
Martijn van Groningen 890d06f018 Added count percolate api
Added a new percolate api that only returns the number of percolate queries that have matched with the document being percolated. The actual query ids are not included. The percolate total count will be put in the total field and is the only result that will be returned from the dedicated count apis.

The total field will also be included in the already existing percolate and percolating existing document apis and are equal to the number of matches.

Closes #3430
2013-08-02 12:30:20 +02:00
Simon Willnauer 2a211705a3 Catch and Log RejectedExecutionException in async ping 2013-08-02 11:32:15 +02:00
Shay Banon a8dcfa5deb Search on a shard group while relocation final flip happens might fail
single shard read operations should have the same override exception logic as search and broadcast

relates to #3427
2013-08-02 09:56:56 +02:00
Alexander Reelsen 343871fcf5 Allow bin/plugin to set -D JVM parameters
Currently the bin/plugin command did not allow one to set jvm parameters
for startup. Usually this parameters are not needed (no need to configure
heap sizes for such a short running process), but one could not set the
configuration path. And that one is important for plugins in order find
out, where the plugin directory is.

This is especially problematic when elasticsearch is installed as
debian/rpm package, because the configuration file is not placed in the
same directory structure the plugin shell script is put.

This pull request allows to call bin/plugin like this

bin/plugin -Des.default.config=/etc/elasticsearch/elasticsearch.yml -install mobz/elasticsearch-head

As a last small improvement, the PluginManager now outputs the directort
the plugin was installed to in order to avoid confusion.

Closes #3304
2013-08-02 09:19:57 +02:00
Shay Banon 235b3a3635 Search on a shard group while relocation final flip happens might fail
make sure relocation shards add their corresponding initializing shard routing when search across initializing shards

also, make shardFailures lazy again

closes #3427
2013-08-02 00:20:10 +02:00
Shay Banon ebda203ce6 less agreesive timeout to catch it on the pending check 2013-08-01 19:52:37 +02:00
Shay Banon 192025401b improve test to wait for nodes before getting the local node id 2013-08-01 19:45:08 +02:00
Shay Banon f3d3a8bd58 Search on a shard group while relocation final flip happens might fail
At the final stage of a relocation, during the final flip of the states, a search request might hit a node that would then execute it on a shard that has already relocated.

For this, we need to execute broadcast and search operations against initializing shards as well, but only as a last resort. The operation will be rejected if not applicable (i.e. IndexShard#searcher() checked for read allowed).

Note, this requires careful though about which failures we send back. If we try and initializing shard and it fails, its failure should not override an actual failure of an active shard.

Also, removed an atomic integer used in broadcast request and use a similar shard index trick we now have in our search execution.

closes #3427
2013-08-01 18:35:58 +02:00
Luca Cavanna 60bddc28eb Modified test to make failures clearer
Added shard failure check when sorting on unmapped field, could be any SearchPhaseExecutionException otherwise (e.g. missing shards)
2013-08-01 17:07:52 +02:00
Simon Willnauer f2f70a415a Take fragile test out of the loop
UpdateNumberOfReplicasTests#simpleUpdateNumberOfReplicasTests is very
fragile due to executing searches based on dated knowledge of
the cluster state and calling shards that have been relocating away in
the mean time. A fix is on the way.
2013-08-01 15:40:04 +02:00
Martijn van Groningen 300db594aa Run refresh before executing non realtime get 2013-08-01 15:12:15 +02:00
Martijn van Groningen a95ce1987e Made use of the static client() method instead of the client field. 2013-08-01 15:07:35 +02:00
Martijn van Groningen 31fd7764e7 If no mapping can be found for value field, throw a proper exception. 2013-08-01 14:09:41 +02:00
Shay Banon 13845e47d6 cleamup ShardIter#reset method
don't have teh reset method return an instance, as it might confuse usage into thinking it might be a different instance
2013-08-01 13:23:39 +02:00
Shay Banon 074b89b7ad better failure message in test 2013-08-01 12:41:42 +02:00
Martijn van Groningen 56227cc141 Improved alias support in the percolate api
* Changed the response to include the alias as part of each match.
* Added `percolate_format=ids` query string option to just serialize the ids in the rest response.
* Added support for multiple indices in the percolate api.

Closes #3420
2013-08-01 10:42:14 +02:00
Alexander Reelsen 5cd01461f6 Improving test CompletionSuggestSearch test stability
Ensuring that the maximum number of replicas is less than the number of nodes.
2013-08-01 10:14:18 +02:00
Alexander Reelsen 4f4f3a2b10 Added prefix suggestions based on AnalyzingSuggester
This commit introduces near realtime suggestions. For more information about
its usage refer to github issue #3376

From the implementation point of view, a custom AnalyzingSuggester is used
in combination with a custom postingsformat (which is not exposed to the user
anywhere for him to use).

Closes #3376
2013-08-01 08:44:09 +02:00
Shay Banon fd15b6278b Query/Filter Facet should support 64bit counter, not 32
closes #3419
2013-07-31 22:10:43 +02:00
Shay Banon e3480a1c0a Reroute eagerly on shard started events
We have an optimization where we try to delay reroute after we processed the shard started events to try and combine a few into the same event. With teh queueing of shard started events in places, we don't need to do it, and we can reroute right away, which will actually reduce the amount of cluster state events we send.

This will also have a nice side effect of not missing on "waitForRelocatingShards(0)" on cluster health checks since relocations will happen right away.

closes #3417
2013-07-31 16:58:26 +02:00
Luca Cavanna 2f8a397aa5 Improved test
- checking routing table taken from same (up-to-date) cluster state
- added @Slow annotation
- forced cluster reroute when needed
- changed order of assertions so that if it fails again it's easier to understand why
2013-07-31 14:26:44 +02:00
Shay Banon 433f0cc86c process deleted index events on a node even if it has no local FS
this will not happen now, but in the future, if data nodes will only be in memory (including translog and such), then we need to fire the deleted events
2013-07-31 13:59:53 +02:00
Adrien Grand 420a3ed691 Forbid usage of StringReader in favor of FastStringReader.
StringReader is synchronized although input streams should always be consumed
by a single thread at a time. FastStringReader on the other hand is completely
thread unsafe.

Closes #3411
2013-07-31 09:34:18 +02:00
Shay Banon 813e3557ab don't call the refresh mapping listener callback twice
though not harmful, results in an annoying transport service log
2013-07-30 23:41:32 +02:00
Peter Moberg 95855c515b auto_expand_replicas causing very large amount of cluster state changes when a node joins or leaves the cluster
Closes #3399
2013-07-30 16:20:18 -04:00
Shay Banon 0e3c67cbf8 Delete API ack to wait also for actual deletion of shards from disk
closes #3413
2013-07-30 18:58:28 +02:00
Boaz Leskes 61036390e2 removed a left over assert made redundant by last commit 2013-07-30 17:39:38 +02:00
Boaz Leskes 925203b9c1 ElasticsearchAssertions.assertHitCount variants now report shard failure upon fail.
Cleaned up a UpdateNumberOfReplicasTests a bit.
2013-07-30 17:22:53 +02:00
Adrien Grand daf7e04caf Use Analyzer.tokenStream(fieldName, text) instead of Analyzer.tokenStream(fieldName, new StringReader(text)).
Lucene 4.4 introduced the tokenStream(String fieldName, String text) helper
method which nicely reuses the string reader. Whenever analyzing Strings, we
should use this method instead of using the Reader method with a
(Fast)StringReader.

Closes #3409
2013-07-30 17:18:08 +02:00
Martijn van Groningen a235a55943 Improved how aliases are handled in the cluster state.
The following changes improved alias creation:
* Moved away from ImmutableMap to JCF's UnmodifiableMap. The ImmutableMap always made a copy, whereas the UnmodifiableMap just wraps the target map.
* Reducing the number of maps being created during the creation of MetaData and IndexMetadata.
* Changed IndexAliasesService's aliases from a copy on write ImmutableMap to ConcurrentMap.

Closes #3410
2013-07-30 17:07:21 +02:00
Shay Banon 33d8571d1e make sure we wrap a potential failure in sending master state change in a catch 2013-07-30 16:44:30 +02:00
Boaz Leskes 668d55758b Added source fetching and filtering parameters to search, get, multi-get, get-source and explain requests
Closes #3301
2013-07-30 13:21:14 +02:00
Luca Cavanna bcbd67feab Solved test timing issue and removed meaningless test 2013-07-30 13:16:57 +02:00
Boaz Leskes 59a9a0d3ef Ported an 0.90 branch excplicit test for dynamic update of gc_delete setting for RobinEngine.
This issue is already solved in master.

See #3396
2013-07-30 12:55:14 +02:00
Shay Banon 07569b4808 warmer test to use 0 replicas
they are meaningless in the context of the test, should stabilize it
2013-07-30 10:47:16 +02:00
Martijn van Groningen 4724725681 Made a clear distinction in the rest api between percolating a provided document and percolating an existing document. The latter is now a separate api. 2013-07-30 10:25:42 +02:00
Martijn van Groningen ef847da2cd Lowered the default size of the percolate thread pool 2013-07-30 09:48:05 +02:00
Martijn van Groningen 7ccc968a9f Moved node level classes classes out of the index.percolator package into toplevel percolator package. 2013-07-29 17:40:07 +02:00
Martijn van Groningen 3f6877ec2b Rewrote the percolate existing doc api.
The percolate existing doc feature now reuses the get request instead of having a separate request body.
Relates to #3380
2013-07-29 17:32:26 +02:00
Martijn van Groningen a9dd3c9756 Added version support to mget and get apis, that only will perform the get operation if the version of the document to be fetched matches with the provided version.
Both get and mget apis now support the following additional parameters: `version` and `version_type`.

Closes #3404
2013-07-29 15:53:12 +02:00
Luca Cavanna 3b2a9fc86b completed renaming of MetaDataStateIndexService to MetaDataIndexStateService 2013-07-29 15:46:50 +02:00
Luca Cavanna 8eed6cdcae Renamed MetaDataStateIndexService to MetaDataIndexStateService for consistency 2013-07-29 15:12:09 +02:00
Luca Cavanna 105d4bab5f Updated test to reflect added acknowledgement from all nodes in open/close index api
If it's acknowledged all the other nodes already hold the latest cluster state, thus we can search on the closed index and be sure we get an error back
2013-07-29 15:12:08 +02:00
Luca Cavanna c23c5d2494 Added support for acknowledgement from other nodes in open/close index api
The open/close index api now waits for an acknowledgement from all the other nodes before returning its response, till the timeout (configurable, default 10 secs) expires. The returned acknowledged flag reflects whether the cluster state change was acknowledged by all the nodes or the timeout expired before.

Closes #3400
2013-07-29 15:12:08 +02:00
Boaz Leskes e0e6a58357 FuzzLikeThis*QueryBuilder and MoreLikeThis*QueryBuilder do not send failOnUnsupportedField if not explicitly specified.
Closes #3402 , a better solution to #3374
2013-07-29 15:02:42 +02:00
Shay Banon 7fd54acd66 remove our own version of highlighting boundary scanner
the bug that we fixed in it was already pushed to the lucene version
2013-07-29 13:36:21 +02:00
Shay Banon fadb0c8f32 cleanup unused classes (not needed with Lucene 4.4) 2013-07-29 12:58:09 +02:00
Shay Banon 2c2cc844dc cleanup deprecations / warnings 2013-07-29 01:55:41 +02:00
Shay Banon 4e66658aac cleanup
- remove Digest/Hex, only used in test, directly use MD5 there
- don't use Closeables and use IOUtils
- remove duplicate Classes usage and unused LongsLAB
2013-07-29 01:29:58 +02:00