Commit Graph

7803 Commits

Author SHA1 Message Date
Martijn van Groningen ade1d0ef57 Added global ordinals (unique incremental numbering for terms) to fielddata.
Added a terms aggregation implementations that work on global ordinals, which is also the default.

Closes #5672
2014-04-07 11:06:41 +07:00
Boaz Leskes 7b9df39800 [Test] Added better control over the number of documents indexed by BackgroundIndexer
Used the new controls to reduce indexing activity in RelocationTests and RecoveryWhileUnderLoadTests

Closes #5696
2014-04-06 20:49:27 +02:00
Shay Banon d26a956231 releasable bytes output + use in transport / translog
create a new releasable bytes output, that can be recycled, and use it in netty and the translog, 2 areas where the recycling will help nicely.
Note, opted for statically typed enforced releasble bytes output, to make sure people take the extra care to control when the bytes reference are released.
 Also, the mock page/array classes were fixed to not take into account potential recycling going during teardown, for example, on a shared cluster ping requests still happen, so recycling happen actively during teardown.
closes #5691
2014-04-06 20:23:21 +02:00
Simon Willnauer a5aafbb04c [TEST] Prevent RelocationTests from going crazy when relocations take time 2014-04-05 22:36:16 +02:00
Simon Willnauer 124d370b5f [TEST] cleanup secondary cluster properly in Tribe tests. 2014-04-05 22:18:38 +02:00
Lee Hinman 211f740100 Add `getAsRatio` to Settings class, allow DiskThresholdDecider to take percentages
Adds new RatioValue class that parses ratios between 0-100% expressed in
either floating-point (0.13) or percentage (51.12%) notation.

Closes #5690
2014-04-04 13:19:35 -06:00
Lee Hinman 3248359660 log full exception in InternalClusterInfoService unless it's a ClusterBlockException 2014-04-04 13:06:01 -06:00
Karl Meisterheim 6d993bc810 [DOCS] A few grammar and word use corrections 2014-04-04 19:26:38 +02:00
Daniel Winterstein 08be94b455 Adding javadoc to UpdateRequestBuilder for a couple of details it took me a while to find. 2014-04-04 17:56:08 +02:00
Peter Dyson 233279bb64 [DOCS] Fixed typo 2014-04-04 17:37:56 +02:00
Timo Rantalaiho 3ae02b0b60 Show stacktrace of startup exception
Whether or not the stacktrace is displayed is controlled by bootstrap
log level setting, so that bootstrap: DEBUG displays the stack trace on
output, like it does on log

Closes #5102
2014-04-04 17:15:41 +02:00
Simon Willnauer 4d6eb369a3 [TEST] Wait for nodes to join before asserting on cluster stats 2014-04-04 14:06:09 +02:00
Boaz Leskes 940954bfd1 [Test] added trace logging to refresh action to note which cluster version was used
Increase logging level for SearchWithRandomExceptionsTests.testRandomExceptions & MinDocCountTests to use the above and log cluster states on changes.
2014-04-04 13:17:28 +02:00
Kevin Wang f582212c68 `geo_point` doesn't allow null values
After upgrading to 1.1.0, sending null values to geo points produces the following error:

```
MapperParsingException[failed to parse]; nested: ElasticsearchParseException[geo_point expected];
```

Closes #5680.
Closes #5681.
2014-04-04 10:53:10 +02:00
Adrien Grand 8a09ec0e06 [TEST] Harden GeoShapeIntegrationTests. 2014-04-04 09:41:07 +02:00
Simon Willnauer df9346dc67 [TEST] build index up-front to prevent relocations 2014-04-03 19:31:22 +02:00
Simon Willnauer cd552e7413 Take stream position into account when calculating remaining length
Currently `PagedBytesReferenceStreamInput#read(byte[],int,int)` ignores
the current pos and that causes to read past EOF

Closes #5667
2014-04-03 17:44:41 +02:00
Adrien Grand 8f324d50b2 [TEST] Reduce number of documents/terms in aggregations' RandomTests.
This test initially had three purposes:
 - duels between equivalent aggregations on significant amounts of data,
 - make sure that array growth works (when the number of buckets grows larger
   than the initial number o buckets),
 - make sure that page recycling works correctly.

Because of the last point, it needed large numbers of docs/terms since page
recycling only kicks in on arrays of more than 16KB. However, since then, we
added a MockPageCacheRecycler to track allocation/release of BigArrays and make
sure that all arrays get released, so we can now lower these numbers of docs/
terms to just make sure that array growth is triggered.
2014-04-03 16:58:44 +02:00
Simon Willnauer 1fc664cc28 [TEST] Fix [Children|Parent]ConstantScoreQueryTests with nightly=true
Those tests use RandomIW which can flush after each document taking forever
to index the 20k docs it was indexing. This commit makes sure the IW is
sane and the number of docs is not too crazy.
2014-04-03 16:25:15 +02:00
Adrien Grand c78bb4472a [TEST] Make sure refresh is called by `indexRandom`, even if the list of documents to index is empty. 2014-04-03 15:57:54 +02:00
Martijn van Groningen 6f0904c7f1 sizeInBytes can be 0 2014-04-03 20:48:50 +07:00
Simon Willnauer 697432390d [TEST] Make BulkTests#testBulkProcessorFlush more robust 2014-04-03 13:32:41 +02:00
Simon Willnauer 6f5b7fa086 [BUILD] Set -Dtests.jvms=auto by default to make use of multiple JVMs 2014-04-03 13:01:01 +02:00
Simon Willnauer 7bc5ab45bc Cleanup IndicesFieldDataCache and IndexFieldDataCache
This commit adds several asserts and removes possible `null` values
from the `FieldDataCache` implementation.

Closes #5664
2014-04-03 12:50:51 +02:00
Martijn van Groningen f389c666c1 Moved the decision to load p/c fielddata eagerly to a better place.
Closes #5569
2014-04-03 14:45:07 +07:00
Lee Hinman c3089701f2 [DOCS] remove extraneous ` from cache page 2014-04-02 16:07:00 -06:00
Alexander Reelsen f0ae43bbea Made template endpoint compatible with search endpoint
Before this the from/size parameters did not work.
Also updated the rest api spec definition file with all the query_string
parameters.

Fixes #5550
2014-04-03 00:01:30 +02:00
Alexander Reelsen e547e113e1 Geo context suggester: Require precision in mapping
The default precision was way too exact and could lead people to
think that geo context suggestions are not working. This patch now
requires you to set the precision in the mapping, as elasticsearch itself
can never tell exactly, what the required precision for the users
suggestions are.

Closes #5621
2014-04-02 23:51:14 +02:00
kul dc19e06e27 Add `flush` method for BulkProcessor class
There is no explicit method `flush/execute` in `BulkProcessor` class. This can be useful in certain scenarios.
Currently it requires to close and create a new BulkProcessor if one wants an immediate flush.

Closes #5575.
Closes #5570.
2014-04-02 19:16:29 +02:00
Britta Weber 0b449d3040 Document es.node.mode and add possible options to error message 2014-04-02 18:32:03 +02:00
Simon Willnauer e189b3e37b [TEST] Use NoMergePolicy rather than NoMergeScheduler in FreqTermsEnumTests 2014-04-02 14:58:45 +02:00
Adrien Grand 4ff3e1926b Remove ScriptDocValues.EMPTY.
Instead the default implementation is used, but on top of empty
(Bytes|Long|Double|GeoPoint)Values. This makes sure there is no
inconsistency between documents depending on whether other documents in the
segment have values or not.

Close #5646
2014-04-02 14:28:42 +02:00
Britta Weber ac5701968b Fix mapping creation on bulk request
When a bulk request triggers an index creation a mapping might not be
created. The reason is that when indexing documents in a bulk,
an indexing operation might fail due to a shard not yet being
started. The mapping service, however, might already
have the mapping but the mapping update is never issued to the master,
even on subsequent indexing of documents.

Instead, the mapping must be propagated to master even if the
indexing fails due to a shard not being started.

closes #5623
2014-04-02 13:53:44 +02:00
Shay Banon 63290a910e Freq Terms Enum
A frequency caching terms enum, that also allows to be configured with an optional filter. To be used by both significant terms and phrase suggester.
This change extracts the frequency caching into the same code, and allow in the future to add a filter to control/customize the background frequencies
Closes #5597
2014-04-02 11:16:00 +01:00
Radu Gheorghe b9cb70198e Typo in the description for include_in_all
I know this is uber-minor, but I was confused by the phrase "the raw field value to be copied". I assume "is" was supposed to be instead of "to"
2014-04-02 12:02:12 +02:00
Boaz Leskes 285fa16f62 [Tests] Reduce the number of threads used by the BackgroundIndexer by default
It sometimes overloads the nodes.
2014-04-02 11:02:38 +02:00
Martijn van Groningen d57ef1e0d6 Removed incorrect comment 2014-04-02 12:14:35 +07:00
Martijn van Groningen b7451533c8 Added an indices level field data cache listener that always gets invoked and updates indices statistics and services about field data loading and unloading.
Moved the circuit breaker memory reducing logic to the IndicesFieldDataCacheListener, so it always reduces the memory used when field data gets unloaded,
this fixes a issue where the circuit breaker didn't get reduced when segments where no shardId could be resolved get cleared up.

Also made sure that exceptions in the percolator service are bubbled up properly.

Closes #5588
2014-04-02 11:55:45 +07:00
Simon Willnauer 42b20d601f Upgrade to Lucene 4.7.1
* Removed XTermsFilter fixed in LUCENE-5502
 * Switched back to automaton queries that caused failures due to LUCENE-5532
 * Fixed Highlight test that has different results due to LUCENE-5538
2014-04-01 23:50:55 +02:00
Binh Ly 51a6a95de3 [DOC] Fixed flags example incorrect syntax 2014-04-01 14:43:38 -04:00
Simon Willnauer c88d32150f [TEST] use length-norm as the tie-breaker in BlendedTermQueryTest 2014-04-01 20:36:43 +02:00
Alexander Reelsen 8b8cd26a59 Geo: Allow to parse lat/lon as strings and coerce them
In order to be more failsafe parsing GeoPoints can support
lat/lon as strings and coerce them. Added support and test for this.
2014-04-01 19:09:33 +02:00
javanna f6bbc894cf [TEST] fixed typo in ReproduceInfoPrinter (s/nighly/nightly) 2014-04-01 18:43:14 +02:00
javanna 6ef45a9ae3 [TEST] Print out the `tests.cluster` sys prop if available to be able to reproduce failures when using the external cluster
Relates to #5630
2014-04-01 18:40:43 +02:00
Shay Banon be33c99835 [TEST] Add ability to start nodes in async mode in TestCluster
this will help speed up when starting multiple nodes in a test
closes #5644
2014-04-01 18:01:47 +02:00
javanna 806c4e87fb [TEST] Removed last occurences of cluster_seed, no longer used
Relates to #5233
2014-04-01 17:57:57 +02:00
Simon Willnauer 9c3b6a50a4 [TEST] Add ThreadLeaks protection to ElasticsearchPostingsFormatTest since we keep running clusters around in the JVM 2014-04-01 17:27:58 +02:00
Simon Willnauer 1f4de9cfb5 Enrich exception message if searcher can not be aquired
This commit also moves the log message after the ensureOpen() call
to only log an error if it's really an erro condition.
2014-04-01 17:13:23 +02:00
Simon Willnauer 2a5ffaafcb Throw EngineClosedException if engine is closed
Currently we throw misleading exception in acquireSearcher
if we try to acquire while we are failing the engine. We should
throw an EngineClosedException instead.

Relates to #5633
2014-04-01 17:00:42 +02:00
Simon Willnauer 257eddd8f0 [TEST] Expect all shards to fail if full refresh failed 2014-04-01 17:00:35 +02:00