Commit Graph

6853 Commits

Author SHA1 Message Date
Simon Willnauer a4f97bed9d Randomize AllocationDecider order in tests 2013-12-17 13:55:43 +01:00
Simon Willnauer 79ab05cdcf Improve allocation of unassigned shards with early termination
When we allocate unassigned shards we can terminate early for some
shards like if we already tried to allocate a replica we don't need
to try the same replica if the first one got rejected. We also
can check if certain nodes can't allocate any primaries or shrads
at all and take those nodes out of the picture for the current round
since it will not change in the current round.
2013-12-17 13:55:43 +01:00
Boaz Leskes 2b6214cff7 Added Cluster Stats API
Closes #4460
2013-12-17 13:14:46 +01:00
Simon Willnauer 75b6415b1a Fail test with timeout and stack dump after 20 min rather than 1h 2013-12-17 12:26:14 +01:00
Grégory Quatannens c64abaae7e Fixing typo and grammar 2013-12-17 11:39:02 +01:00
Boaz Leskes 9fb361cea1 Move index health calculations to ClusterIndexHealth so it can be reused. 2013-12-17 11:31:22 +01:00
Adrien Grand 33599d9a34 Compressed geo-point field data.
This commit allows to trade precision for memory when storing geo points.
This new field data impl accepts a `precision` parameter that controls the
maximum expected error for storing coordinates. This option can be updated on
a live index with the PUT mapping API.

Default precision is 1cm, which requires 8 bytes per geo-point (50% memory
saving compared to using 2 doubles).

Close #4386
2013-12-17 11:29:48 +01:00
Clinton Gormley 684affa5c7 [DOCS] Removed unused file 2013-12-17 11:28:19 +01:00
Shay Banon a1ee68a145 fix usage of deprecated netty header method 2013-12-17 10:59:27 +01:00
Alexander Reelsen b713cf56ed Allow to provide parameters not only through -D but as long parameters
All getopt long style parameters are now set as es. properties,

elasticsearch --path.data=/some/path

results in -Des.path.data=/some/path

Closes #4393
2013-12-17 10:43:27 +01:00
Alexander Reelsen c30945a3d8 Start elasticsearch in the foreground by default
Instead of using the '-f' parameter to start elasticsearch in the
foreground, this is now the default modus.

In order to start elasticsearch in the background, the '-d' parameter
can be used.

Closes #4392
2013-12-17 10:39:22 +01:00
Shay Banon 809e870b8d introduce a native int/long open immutable map, and use it in in routing table 2013-12-16 20:33:38 +01:00
Simon Willnauer 1dc8c079da Wait until index is in the clusterstate after restart 2013-12-16 19:52:00 +01:00
Clinton Gormley 34b9b16233 [DOCS] Fixed some bad link refs 2013-12-16 18:07:33 +01:00
Martijn van Groningen 23d2b1ea7b Renamed top level `filter` to `post_filter`.
Closes #4119
2013-12-16 17:10:14 +01:00
Lee Hinman db431b7cb3 Remove the `field` and `text` queries.
The `text` query was replaced by the `match` query and has been
deprecated for quite a while.

The `field` query should be replaced by a `query_string` query with
the `default_field` specified.

Fixes #4033
2013-12-16 08:59:36 -07:00
Simon Willnauer 3e321972cc Throw IAE if suggest results return differently sized results.
If the term suggester is used the results are merged depending on
the number of terms produced by the tokenizer / tokenfilter. If a
term suggester is executed across multiple indices that share the
same field but with different analysis chains we can't merge the
result anymore sicne tokens are our of order or have a different size.

This commit throws ESIllegalArgumentException if the number of entries
are not the same across all results.

Closes #3196
2013-12-16 15:31:44 +01:00
Shay Banon 2f2b95a6b8 better cluster reroute allocation benchmark 2013-12-16 15:29:50 +01:00
Luca Cavanna 173a91bb46 Added new IndicesLifecycle.Listener method that allows to listen for any IndexShardState internal change.
Closes #4413
2013-12-16 15:00:15 +01:00
Adrien Grand 4e7ce4ee02 Make field data changes immediately taken into account and add the ability to disallow field data loading.
This commit changes field data configuration updates so that they are
immediately taken into account for loading new segments. The way it works
is that field data configuration is now cached separately from the field
data cache, meaning that it is now possible to clear the field data
configuration from IndexFieldDataService while the cache will stay around. On
the next time that Elasticsearch will reload field data configuration, it will
check if there is already a cache entry, and reuse it if it exists.

To disable field data loading, all that is required is to change the field
data format to "none" (supported by all field data types) using the update
mapping API. Elasticsearch will then refuse to load field data on any new
segment, but field data which has been loaded on the previous segments will
remain available. So you need to clear the field data cache in order to
reclaim memory (otherwise memory will be reclaimed slower, as segments get
merged).

Close #4430
Close #4431
2013-12-16 14:34:33 +01:00
Simon Willnauer 8d321530de Reset source shards to `started` if canceling relocation.
Currently we miss to reset the source shards status to ACTIVE if we cancel
a relocation. If the shard is RELOCATING we need to reset to state ACTIVE.

Closes #4457
2013-12-16 11:52:16 +01:00
Simon Willnauer 30c6f2fa23 Improve RoutingNodes API
Currently the RoutingNodes API allows modification of it's internal state outside of the class.
This commit improves the APIs of `RoutingNode` and `RoutingNode` to change internal state
only within the classes itself.

Closes #4458
2013-12-16 11:50:45 +01:00
Sebastian Geidies 6af80d5017 Optimizes performance of AllocationDecider execution. Instead of using loops over all ShardRoutings, do accounting in RoutingNodes.
Speeds up recalculating cluster state on large clusters.
2013-12-16 11:35:45 +01:00
Alexander Reelsen 6a856c86e8 Cat API: Add endpoint to show aliases
This endpoint allows to check aliases, their indices, if a filter is
configured along with routing values for searching and indexing.

Closes #4414
2013-12-16 10:37:06 +01:00
Boaz Leskes 33e49a55f7 Increased timeout in ClusterServiceTests.testTimeoutUpdateTask to 100ms
The previous 2ms timeout was too small and caused a racing condition in timeout handling. This will be fixed but requires more work.
2013-12-16 10:02:24 +01:00
Simon Willnauer a668cd22d8 wait until cluster-state is ready after startup 2013-12-15 22:26:11 +01:00
Simon Willnauer d9e988aea9 Fix QuorumLocalGatewayTests#testChangeInitialShardsRecovery to start only one shard to not fulfill the quorum 2013-12-15 20:06:22 +01:00
Boaz Leskes 0f7c5dbc86 Update minimum_master_nodes comment to match documentation
Someone on the mailing list wrongly interpreted it to mean the current node is not included in the calculation.
2013-12-15 11:25:36 +01:00
Igor Motov d8ba92cfa8 Resolve potential deadlock state during EsThreadPoolExecutor shutdown
Fixes #4334

The deadlock occurs between monitor object of EsThreadPoolExecutor and mainLock of ThreadPoolExecutor. The shutdown method of EsThreadPoolExecutor obtains the lock on monitor first and waits for mainLock of ThreadPoolExecutor in ThreadPoolExecutor#shutdown for part of the processing, while EsThreadPoolExecutor#terminated is executed under mainLock and tries to obtain monitor to notify listeners.
2013-12-14 21:45:03 -05:00
Simon Willnauer 8f85d63b67 Enable incremental compilation using a workaround for the maven-compiler-plugin 3.1 bug 2013-12-14 21:56:01 +01:00
Shay Banon 80ab75e54e upgrade to latest jsr166 2013-12-14 01:27:18 +01:00
Shay Banon 59d85bc22a upgrade to Jackson 2.3.0 2013-12-14 00:53:52 +01:00
Luca Cavanna 9b121baafc More common AcknowledgedRestResponseActionListener
Introduced use of AcknowledgedRestResponseActionListener in put/delete index template
2013-12-13 19:39:46 +01:00
Simon Willnauer d684a2b8da Don't try to run check-index in MockDirectoryWrapper
if the MockDirWrapper checks the index on close it closes all
closeables that is has to crash the index if something is not flushed yet.
For us this is a problem since the input is still used. We need to fix this in
lucene first.
2013-12-13 17:44:10 +01:00
Adrien Grand 19c8d18b14 Relax a bit the accuracy tests on SloppyMaths.
Tests now ensure that the computed distance is correct within 1‰ instead of 1‱.
2013-12-13 15:42:39 +01:00
Adrien Grand 36bd9cc432 Aggregations: Ordinals-based string bucketing support.
When the ValuesSource has ordinals, terms ordinals are used as a cache key to
bucket ordinals. This can make terms aggregations on String terms significantly
faster.

Close #4350
2013-12-13 15:34:02 +01:00
Martijn van Groningen cc61348013 Fixed test bug 2013-12-13 14:54:21 +01:00
Martijn van Groningen 10e2528cce Added the `force_source` option to highlighting that enforces to use of the _source even if there are stored fields.
The percolator uses this option to deal with the fact that the MemoryIndex doesn't support stored fields,
this is possible b/c the _source of the document being percolated is always present.

Closes #4348
2013-12-13 13:39:53 +01:00
Simon Willnauer 59e4e58683 Add trace logging to AllocationDeciders if decision is NO() 2013-12-13 12:08:26 +01:00
Sebastian Geidies 6ed126deaf Skipping execution of remaining Deciders if one of them returns a Decision.NO.
Reordering execution of the different AllocationDeciders, so the cheapest get executed first.
2013-12-13 11:52:48 +01:00
Boaz Leskes 2acccf257d Add `queries` to XContent output of PercolateStats
The queries stats tracked but not exposed to the Rest API

Closes #4440
2013-12-13 11:36:51 +01:00
Boaz Leskes b65d07b495 A failure during the response construction could cause a Node Operation never to return 2013-12-13 10:36:39 +01:00
Diego Ongaro a4814c2f69 Print nice error in bin/elasticsearch if user needs to run maven
Before, people that cloned the repo and expected to be able to run
bin/elasticsearch would be met with an awful shell error: the shell
interprets Maven variables like ${project.build.finalName} as shell
variables yet can't handle names of the form ${x.y}. This commit
explicitly checks to make sure that Maven has done its substitutions
before continuing; if Maven hasn't been run, it gives a helpful error
message.

Fixes #2954.
2013-12-13 10:30:42 +01:00
Shay Banon 976bdd6a26 randomize transport tcp compression on nodes 2013-12-12 23:19:59 +01:00
Shay Banon 0469cca4be properly support global transport compression with new bytes transport request optimization 2013-12-12 22:29:54 +01:00
Shay Banon 0f4d81fcd4 refresh the mapping source directly into a compressed buffer
instead of building the mapping source, and then compressing it, we can generate it directly into a compressed buffer
2013-12-12 20:58:11 +01:00
Lee Hinman 77fcf71338 Add new `simple_query_string` query type
This adds support for Lucene's SimpleQueryParser by adding a new type
of query called the `simple_query_string`. The `simple_query_string`
query is designed to be able to parse human-entered queries without
throwing any exceptions.

Resolves #4159
2013-12-12 12:09:32 -07:00
Alexander Reelsen 348c3731c8 Fixing debian init script to work on ubuntu LTS
sysctl parameters must be separated like `-q -w` instead of `-qw`
2013-12-12 17:57:34 +01:00
Adrien Grand e2e19203d7 Remove IndexFieldData.getHighestNumberOfSeenUniqueValues().
Closes #4426
2013-12-12 15:44:05 +01:00
Shay Banon f0e1f9215b try and wait better for all mapping updates to be processes in test 2013-12-12 15:06:37 +01:00