Commit Graph

7776 Commits

Author SHA1 Message Date
Alexander Reelsen e547e113e1 Geo context suggester: Require precision in mapping
The default precision was way too exact and could lead people to
think that geo context suggestions are not working. This patch now
requires you to set the precision in the mapping, as elasticsearch itself
can never tell exactly, what the required precision for the users
suggestions are.

Closes #5621
2014-04-02 23:51:14 +02:00
kul dc19e06e27 Add `flush` method for BulkProcessor class
There is no explicit method `flush/execute` in `BulkProcessor` class. This can be useful in certain scenarios.
Currently it requires to close and create a new BulkProcessor if one wants an immediate flush.

Closes #5575.
Closes #5570.
2014-04-02 19:16:29 +02:00
Britta Weber 0b449d3040 Document es.node.mode and add possible options to error message 2014-04-02 18:32:03 +02:00
Simon Willnauer e189b3e37b [TEST] Use NoMergePolicy rather than NoMergeScheduler in FreqTermsEnumTests 2014-04-02 14:58:45 +02:00
Adrien Grand 4ff3e1926b Remove ScriptDocValues.EMPTY.
Instead the default implementation is used, but on top of empty
(Bytes|Long|Double|GeoPoint)Values. This makes sure there is no
inconsistency between documents depending on whether other documents in the
segment have values or not.

Close #5646
2014-04-02 14:28:42 +02:00
Britta Weber ac5701968b Fix mapping creation on bulk request
When a bulk request triggers an index creation a mapping might not be
created. The reason is that when indexing documents in a bulk,
an indexing operation might fail due to a shard not yet being
started. The mapping service, however, might already
have the mapping but the mapping update is never issued to the master,
even on subsequent indexing of documents.

Instead, the mapping must be propagated to master even if the
indexing fails due to a shard not being started.

closes #5623
2014-04-02 13:53:44 +02:00
Shay Banon 63290a910e Freq Terms Enum
A frequency caching terms enum, that also allows to be configured with an optional filter. To be used by both significant terms and phrase suggester.
This change extracts the frequency caching into the same code, and allow in the future to add a filter to control/customize the background frequencies
Closes #5597
2014-04-02 11:16:00 +01:00
Radu Gheorghe b9cb70198e Typo in the description for include_in_all
I know this is uber-minor, but I was confused by the phrase "the raw field value to be copied". I assume "is" was supposed to be instead of "to"
2014-04-02 12:02:12 +02:00
Boaz Leskes 285fa16f62 [Tests] Reduce the number of threads used by the BackgroundIndexer by default
It sometimes overloads the nodes.
2014-04-02 11:02:38 +02:00
Martijn van Groningen d57ef1e0d6 Removed incorrect comment 2014-04-02 12:14:35 +07:00
Martijn van Groningen b7451533c8 Added an indices level field data cache listener that always gets invoked and updates indices statistics and services about field data loading and unloading.
Moved the circuit breaker memory reducing logic to the IndicesFieldDataCacheListener, so it always reduces the memory used when field data gets unloaded,
this fixes a issue where the circuit breaker didn't get reduced when segments where no shardId could be resolved get cleared up.

Also made sure that exceptions in the percolator service are bubbled up properly.

Closes #5588
2014-04-02 11:55:45 +07:00
Simon Willnauer 42b20d601f Upgrade to Lucene 4.7.1
* Removed XTermsFilter fixed in LUCENE-5502
 * Switched back to automaton queries that caused failures due to LUCENE-5532
 * Fixed Highlight test that has different results due to LUCENE-5538
2014-04-01 23:50:55 +02:00
Binh Ly 51a6a95de3 [DOC] Fixed flags example incorrect syntax 2014-04-01 14:43:38 -04:00
Simon Willnauer c88d32150f [TEST] use length-norm as the tie-breaker in BlendedTermQueryTest 2014-04-01 20:36:43 +02:00
Alexander Reelsen 8b8cd26a59 Geo: Allow to parse lat/lon as strings and coerce them
In order to be more failsafe parsing GeoPoints can support
lat/lon as strings and coerce them. Added support and test for this.
2014-04-01 19:09:33 +02:00
javanna f6bbc894cf [TEST] fixed typo in ReproduceInfoPrinter (s/nighly/nightly) 2014-04-01 18:43:14 +02:00
javanna 6ef45a9ae3 [TEST] Print out the `tests.cluster` sys prop if available to be able to reproduce failures when using the external cluster
Relates to #5630
2014-04-01 18:40:43 +02:00
Shay Banon be33c99835 [TEST] Add ability to start nodes in async mode in TestCluster
this will help speed up when starting multiple nodes in a test
closes #5644
2014-04-01 18:01:47 +02:00
javanna 806c4e87fb [TEST] Removed last occurences of cluster_seed, no longer used
Relates to #5233
2014-04-01 17:57:57 +02:00
Simon Willnauer 9c3b6a50a4 [TEST] Add ThreadLeaks protection to ElasticsearchPostingsFormatTest since we keep running clusters around in the JVM 2014-04-01 17:27:58 +02:00
Simon Willnauer 1f4de9cfb5 Enrich exception message if searcher can not be aquired
This commit also moves the log message after the ensureOpen() call
to only log an error if it's really an erro condition.
2014-04-01 17:13:23 +02:00
Simon Willnauer 2a5ffaafcb Throw EngineClosedException if engine is closed
Currently we throw misleading exception in acquireSearcher
if we try to acquire while we are failing the engine. We should
throw an EngineClosedException instead.

Relates to #5633
2014-04-01 17:00:42 +02:00
Simon Willnauer 257eddd8f0 [TEST] Expect all shards to fail if full refresh failed 2014-04-01 17:00:35 +02:00
Simon Willnauer 4da66c4f95 ignore index if it's not in the cluster state and it's in the drop indices set
Closes #5643
2014-04-01 16:02:43 +02:00
Simon Willnauer 98aecee531 [TEST] use assertAcked when creating indices 2014-04-01 16:02:30 +02:00
Robert Muir 24005b4011 add test for ElasticSearchPostingsFormat 2014-04-01 08:50:22 -04:00
Simon Willnauer 268f2c3b12 [TEST] Fix BulkTests#testThatInvalidIndexNamesShouldNotBreakCompleteBulkRequest - randomBoolean() doesn't always return true 2014-04-01 14:38:35 +02:00
Shay Banon c83c72b165 Fail the engine/shard when refresh failed
When refresh failed, it would fail due to a serious issue in the shard (mainly corrupted index). Fail the engine in that cause, which will cause the shard to fail. The reason why its not only on CorruptedIndex failed is that any type of failure seems to be relevant here to fail the shard
closes #5633
2014-04-01 14:32:20 +02:00
Igor Motov d13850814e [DOCS] "F" is not valid false value for boolean type 2014-04-01 08:16:43 -04:00
Simon Willnauer 71de2bc414 [BUILD] Allow to set tests memory via the commandline 2014-04-01 14:12:52 +02:00
Boaz Leskes 83a013320c Moved BackgroundIndexer to a top level class. Moved waitNumDocs to ElasticsearchIntegrationTestBase.
Update RelocationTests to use the above and unified testPrimaryRelocationWhileIndexing & testReplicaRelocationWhileIndexingRandom into a single randomized test.
2014-04-01 14:12:52 +02:00
Adrien Grand a6a12f97a2 [Test] allow tests to reuse a singel index across tests
This commit speeds up aggregations tests dramatically since they all
rely on the same index structure. Here we can safe a large amout of time
to not recreate the index for each small test.
2014-04-01 14:12:45 +02:00
Boaz Leskes 5bf8b79587 Add a BackgroundIndexer class to RecoveryWhileUnderLoadTests and use it.
Also change the document distribution a bit between the tests
2014-04-01 12:03:57 +02:00
Simon Willnauer e621458a39 [Test] Speedup UpdateTests#testConcurrentUpdateWithRetryOnConflict 2014-04-01 12:03:57 +02:00
Simon Willnauer 6898984307 [Test] Speeup FieldData tests and remove unnecessary @Repeat annotations 2014-04-01 12:03:57 +02:00
Simon Willnauer 6c3ec9e719 [Test] Run intensive suggest tests only if nightly=true 2014-04-01 12:03:57 +02:00
Simon Willnauer e3317f2c6b [Test] Speedup RecoveryWhileUnderLoadTests to run with less documents unless tests.nightly=true 2014-04-01 12:03:57 +02:00
Simon Willnauer e60ca8d8d7 [Test] Speed up RandomAllocationDeciderTests 2014-04-01 12:03:57 +02:00
Shay Banon 6e12584018 [TEST] delegate to the original transport 2014-04-01 11:52:18 +02:00
javanna 38dd501ab5 [TEST] added ExternalTestCluster that allows to run tests against an external cluster
All the ordinary test operations happen based on the ImmutableTestCluster base class and are executed via transport client. Will be used especially for the REST tests once migrated to the standard randomized runner.

Added new httpAddresses method to ImmutableTestCluster to be able to retrieve the http addresses to connect to for the REST tests. Both versions will look inside the cluster to figure out which nodes are available for http calls and their addresses.

The external cluster is used as global cluster if the tests.cluster system property is available. The property needs to contain a comma separated list of available elasticsearch nodes that will be used to connect to the cluster (e.g. localhost:9300,localhost:9301).

Only a subset of the integration tests can currently be run successfully against the external cluster, for more precision the ones that don't modify the cluster layout (don't require cluster() functionalities but rely only on immutableCluster()). Also at least two data nodes are required otherwise the ensureGreen calls cannot succeed.

Closes #5630
2014-04-01 11:45:35 +02:00
Chris Earle 70c089de0a Replaces usage of `StringBuffer` with `StringBuilder` and fixes stray
single quote in the same `PluginInfo#toString()` method.

Closes #5605
2014-04-01 10:40:07 +02:00
Adrien Grand a34378f852 Do not propagate errors from onResult to onFailure.
The way that SearchServiceTransportAction executes actions on a local node
today would propagate exceptions thrown in onResult to onFailure.

Close #5629
2014-04-01 10:02:29 +02:00
Bill Hwang fac4da822d [BUILD] Switch 1x to JDK7 download
Changed download URL for 1x build to point to JDK7 latest
2014-03-31 19:28:27 -07:00
Boaz Leskes 350fd8e30b Added trace logging in TransportSearchTypeAction and a delayed verification of test failures in MinDocCountTests.testMinDocCountOnTerms 2014-03-31 23:23:46 +02:00
Shay Banon cc4cae3ba0 Mock Transport: Allow to simulate network failures
An infrastructure that allows to simulate different network topologies failures, including 2 basic ones in failure to send requests, and unresponsive nodes
closes #5631
2014-03-31 22:49:10 +02:00
Boaz Leskes 62b850f52d InternalEngine - back to set dirty=false just *before* a refresh is done
reverting: 5553c383bc
2014-03-31 20:57:58 +02:00
Simon Willnauer c7b1f9946f Discard new cluster state if the clustername doesn't match
Closes #5622
2014-03-31 20:23:49 +02:00
Simon Willnauer df7474b9fc Add cluster_name to cluster_state
Today the clusterstate is not asssociated with the cluster_name which is odd
since it's pretty much it's only valid identifier. Any node can send a cluster
state to another node today even if it's not the same cluster. These situations
can happen rarely during tests or even in the wild by accident.
2014-03-31 20:23:49 +02:00
Nik Everett 1df942b463 [docs] Indices stats groups in nodes api
Closes #5349
2014-03-31 19:54:48 +02:00
javanna d570d588a8 [TEST] moved dataNodes method to ImmutableTestCluster base class, replaced cluster().dataNodes() calls with immutableCluster().dataNodes() 2014-03-31 19:09:53 +02:00