4196 Commits

Author SHA1 Message Date
Adrien Grand
01327d7136 Facets: deprecation.
Users are encouraged to move to the new aggregation framework that was
introduced in Elasticsearch 1.0.

Close #6485
2014-06-13 13:13:44 +02:00
Martijn van Groningen
59ff05020f [TEST] Removed incorrect assertion (it is expected that the flush doesn't execute on all shard copies, because we don't wait for green status) 2014-06-13 12:25:43 +02:00
mikemccand
9620aa315e [TEST] Add FailureMarker to test listeners so -Dtests.failfast works 2014-06-13 06:04:33 -04:00
Martijn van Groningen
77e0429089 [TEST] Verify the flush reponse 2014-06-13 11:40:05 +02:00
Boaz Leskes
7fb16c783d Added caching support to geohash_filter
Caching is turned off by default.

Closes #6478
2014-06-12 22:19:34 +02:00
Shay Banon
2330421816 Wait till node is part of cluster state for join process
When a node sends a join request to the master, only send back the response after it has been added to the master cluster state and published.
This will fix the rare cases where today, a join request can return, and the master, since its under load, have not yet added the node to its cluster state, and the node that joined will start a fault detect against the master, failing since its not part of the cluster state.
Since now the join request is longer, also increase the join request timeout default.
closes #6480
2014-06-12 18:15:51 +02:00
Lee Hinman
3a3f81d59b Enable DiskThresholdDecider by default, change default limits to 85/90%
Fixes #6200
Fixes #6201
2014-06-12 16:35:29 +02:00
Alex Ksikes
35cba50fce More Like This Query: creates only one MLT query per field for all queried items.
Previously, one MLT query per field was created for each item. One issue with
this method is that the maximum number of selected terms was equal to the
number of items times 'max_query_terms'. Instead, users should have direct control
over the maximum number of selected terms allowed, regardless of the number of
queried items.

Another issue related to the previous method is that it could lead to the
selection of rather uninteresting terms, that because they were found in a
particular queried item. Instead, this new procedure enforces the selection of
interesting terms across ALL items, not within each item. This could lead to
search results where the best matching items share commonalities amongst the
best characteristics of all the items.

Closes #6404
2014-06-12 14:19:33 +02:00
Simon Willnauer
5575ba1a12 [BUILD] Check for tabs and nocommits in the code on validate
This commit adds checks for nocommit and tabs in the source code.
The task is executed during the validate phase and can be disabled via
`-Dvalidate.skip`
2014-06-12 11:11:23 +02:00
Israel Tsadok
32f87f8617 Highlighting: make HighlightQuery class public 2014-06-11 17:23:43 +02:00
Clinton Gormley
673ef3db3f The StemmerTokenFilter had a number of issues:
* `english` returned the slow snowball English stemmer
* `porter2` returned the snowball Porter stemmer (v1)
* `portuguese` was used twice, preventing the second version from working

Changes:

* `english` now returns the fast PorterStemmer (for indices created from v1.3.0 onwards)
* `porter2` now returns the snowball English stemmer (for indices created from v1.3.0 onwards)
* `light_english` now returns the `kstem` stemmer (`kstem` still works)
* `portuguese_rslp` returns the PortugueseStemmer
* `dutch_kp` is a synonym for `kp`

Tests and docs updated

Fixes #6345
Fixes #6213
Fixes #6330
2014-06-11 12:30:16 +02:00
Clinton Gormley
c25de57d5d Tests: Fixed CompletionSuggester test which relied on a bug 2014-06-10 21:34:03 +02:00
Clinton Gormley
bb15def36e Stats: Bugfixes and enhancements to indices stats API
Bugs:
* "groups" and "types" were being ignored
* "completion_fields" as wildcards were not being resolved to fieldnames

Enhancements:
* Made "groups" and "types" support wildcards
* Added missing tests

Closes #6390
2014-06-10 17:35:49 +02:00
Alexander Reelsen
d3dc158458 TransportClient: Improve logging, fix minor issue
In order to return more information to the client, in case a TransportClient
can not connect to the cluster, this commit adds logging and also returns the
configured nodes in the NoNodeAvailableException

Also a minor bug has been fixed, which propagated exceptions wrong, so that an
invalid request was actually tried on every node, if a regular connection failure
on the first node had happened.

Closes #6376
2014-06-10 13:15:59 +02:00
Martijn van Groningen
38be1e0dde Aggregations: if maxOrd is 0 then use noop collector
Before the OrdinalsCollector was used and this leads to a ArrayIndexOutOfBoundsException

Closes #6413
2014-06-10 09:14:06 +02:00
Martijn van Groningen
e15d2e2514 Fielddata: EmptyOrdinals#getMaxOrd() should return 0 instead of 1, since ordinals are zero based since #5871. 2014-06-10 09:13:27 +02:00
Martijn van Groningen
5e408f3d40 Change the top_hits to be a metric aggregation instead of a bucket aggregation (which can't have an sub aggs)
Closes #6395
Closes #6434
2014-06-10 09:09:50 +02:00
javanna
ed5b49a5be [TEST] Added backwards compatibility check to control whether to enable client nodes or not within TestCluster
Our REST backwards compatibility tests need to be able to disable client nodes within the TestCluster when running older tests that assume client nodes are not around.
2014-06-07 15:39:56 +02:00
mikemccand
bb8a666b6d make test less evil 2014-06-07 04:15:52 -04:00
Boaz Leskes
a06b84d392 [Test] Enabled trace logging to testAutoGenerateIdNoDuplicates
also increased iterations some, to increase chance of identifying bad shards
2014-06-07 09:47:15 +02:00
Boaz Leskes
6c7d260770 fixing recovery debug logging param mismatch 2014-06-07 09:36:48 +02:00
Matthew L Daniel
b0a85f6ca3 Guard against improper auto_expand_replica values
Previously if the user provided a non-conforming string, it would blow up with
`java.lang.StringIndexOutOfBoundsException: String index out of range: -1`
which is not a *helpful* error message.

Also updated the documentation to make the possible setting values more clear.

Close #5752
2014-06-07 01:19:06 +02:00
Boaz Leskes
b454f64c57 Bulk request which try and fail to create multiple indices may never return
This is caused by an NPE in the error handling code. All is well if only 1 index creation fails (or none).

Closes #6436
2014-06-06 23:10:42 +02:00
markharwood
724129e6ce Aggregations optimisation for memory usage. Added changes to core Aggregator class to support a new mode of deferred collection.
A new "breadth_first" results collection mode allows upper branches of aggregation tree to be calculated and then pruned
to a smaller selection before advancing into executing collection on child branches.

Closes #6128
2014-06-06 15:59:51 +01:00
javanna
11f7c31852 Put index template api: unified PUT/POST behaviour in relation to create parameter
The put index template api supports the create parameter (defaults to false), which tells whether the template can replace an existing one with same name or not. Unified its behaviour between PUT and POST method, whereas the POST would previously force create to true.

Added create parameter to the rest spec (was missing before) and a REST test for create true scenario.
2014-06-06 15:45:05 +02:00
Simon Willnauer
797a9b07ef FileSystem: Use XNativeFSLockFactory instead of the buggy Lucene 4.8.1 version
There is a pretty nasty bug in the lock factory we use that can cause
nodes to use the same data dir wiping each others data. Luckily this is
unlikely to happen if the nodes are running in different JVM which they
do unless they are embedded.

See LUCENE-5738

Closes #6424
2014-06-06 11:51:47 +02:00
mikemccand
2a6468efbd make this new test a bit less stressful for nightly; catch FlushNotAllowedEngineException 2014-06-05 13:57:59 -04:00
mikemccand
59635f9397 Core: switch to the new ConcurrentHashMap implementation coming in Java 8
The new implementation has lower RAM overhead and better concurrency
in some cases.

Closes #6400
2014-06-05 13:39:23 -04:00
stephlag
6a82d59cb8 [DOCS] Added Javadocs to ESLogger and ESLoggerFactory 2014-06-05 19:15:22 +02:00
mikemccand
30d8467775 revert CHMV8 for now (it doesn't compile under Java8) 2014-06-05 12:13:06 -04:00
javanna
21772e0bf9 Scripts: exposed _uid, _id and _type fields as stored fields (_fields notation)
The _uid field wasn't available in a script despite it's always stored. Made it available and made available also _id and _type fields that are deducted from it.

Closes #6406
2014-06-05 17:16:55 +02:00
mikemccand
838142646f Core: switch to the new ConcurrentHashMap implementation coming in Java 8
The new implementation has lower RAM overhead and better concurrency
in some cases.

Closes #6400
2014-06-05 10:49:23 -04:00
mikemccand
2ad8a60532 add versioning test 2014-06-05 09:38:22 -04:00
stephlag
b5c9d8c98b Add Javadoc 2014-06-04 17:18:25 +02:00
mikemccand
50e42265ef Indexing: clear versionMap on refresh (not flush) to reduce heap usage
The versionMap holds all versions (keyed by _uid) for recently indexed
documents.  Previously we only cleared it during flush, which can be
infrequent if the translog flush thresholds are high, and can cause
excessive heap usage especially for small documents.

Now we clear it during refresh which is usually more frequent
(e.g. once per second by default).

Closes #6379
2014-06-04 05:37:51 -04:00
Colin Goodheart-Smithe
f78480a0bc Aggregations: Fixed failures when geo points are all either positive or negative 2014-06-04 09:16:29 +01:00
Simon Willnauer
288eb3d803 [TEST] remove trace logging 2014-06-04 10:10:38 +02:00
Boaz Leskes
ef5d64c73b [Test] Extended IndexActionTests.testAutoGenerateIdNoDuplicates to check both with and without a specific type
The test also captures the first error but continues to run searches in order to gather more information before failing.
2014-06-03 21:55:10 +02:00
Simon Willnauer
963f627dca Add [1.2.1] Release 2014-06-03 17:25:57 +02:00
Colin Goodheart-Smithe
b9f4d44b14 Aggregations: Adds GeoBounds Aggregation
The GeoBounds Aggregation is a new single bucket aggregation which outputs the coordinates of a bounding box containing all the points from all the documents passed to the aggregation as well as the doc count. Geobound Aggregation also use a wrap_logitude parameter which specifies whether the resulting bounding box is permitted to overlap the international date line.  This option defaults to true.

This aggregation introduces the idea of MetricsAggregation which do not return double values and cannot be used for sorting.  The existing MetricsAggregation has been renamed to NumericMetricsAggregation and is a subclass of MetricsAggregation.  MetricsAggregations do not store doc counts and do not support child aggregations.

Closes #5634
2014-06-03 15:59:56 +01:00
Simon Willnauer
4b28bc396d Translog: Revert unlimited flush_treshold_ops for translog
This commit reverts the commit for issue #5900 introduced
in `1.2.0`. The unlimited translog size can cause memory pressure
on ES instances with low memory and high indexing load.

Closes #6377
2014-06-03 16:54:22 +02:00
Adrien Grand
7ab99de483 Routing: Restore shard routing.
Routing has been inadvertly changed in #5562 resulting in documents going to
different shards in 1.2. This is a terrible bug because an indexing request
would not necessarily go to the same shard anymore, potentially leading to
duplicates.

Close #6391
2014-06-03 16:37:54 +02:00
Kevin Wang
6a399d4c9a Remove support for field names in node_stats url
Field names ended up making the urls too long, fields are still supported as query string parameters though (same as indices stats)
2014-06-03 13:57:07 +02:00
stephlag
10cb136eb0 [DOCS] Fixed typo in IndexRequestBuilder Javadocs 2014-06-03 13:48:41 +02:00
Alex Ksikes
9797e343aa More Like This Query: values of a multi-value fields are compared at the same level.
Previously, More Like This would create a new mlt query for each value of a
multi-value field. This could result in all the values of the field to be
selected, which defeats the purpose of More Like This. Instead, the correct
behavior is to generate only one mlt query for all the values of the field.
This commit provides the correct behavior for More Like This DSL. The fix for
More Like This API will be coming in another commit.

Closes #6310
2014-06-03 13:43:51 +02:00
Adrien Grand
df67b17646 BigArrays: Disable breaking.
The BigArrays limit is currently shared by the translog, netty, http and some
queries/aggregations. If any of these consumers starts taking a lot of memory,
then other ones might fail to allocate memory, which could have bad
consequences, eg. if ping requests can't be sent. The plan is to come up with
a better solution in 1.3.

Close #6332
2014-06-03 11:34:25 +02:00
javanna
90b1e6a461 [TEST] make sure that the -Dtests.rest.blacklist parameter works on windows too
Some reserved characters need to be replaced in the test section names, which gets parsed as a path although it isn't a filename
2014-06-03 09:23:37 +02:00
Britta Weber
125e0c16cd Object and Type parsing: Fix include_in_all in type
include_in_all can also be set on type level (root object).
This fixes a regression introduced  in #6093

closes #6304
2014-06-02 17:48:19 +02:00
Colin Goodheart-Smithe
a23e4aefaa Geo: Issue with polygons near date line
If a polygon is constructed which overlaps the date line but has a hole which lies entirely one to one side of the date line, JTS error saying that the hole is not within the bounds of the polygon because the code which splits the polygon either side of the date line does not add the hole to the correct component of the final set of polygons.  The fix ensures this selection happens correctly.

Closes #6179
2014-06-02 15:03:32 +01:00
Martijn van Groningen
f2641d29ae [TEST] Added sort duel between a single shard index and a multi shard index. 2014-06-02 14:16:55 +02:00