Commit Graph

1920 Commits

Author SHA1 Message Date
Lee Hinman 7d52d58747 Add AllocationDecider that takes free disk space into account
This commit adds two main pieces, the first is a ClusterInfoService
that provides a service running on the master nodes that fetches the
total/free bytes for each data node in the cluster as well as the
sizes of all shards in the cluster. This information is gathered by
default every 30 seconds, and can be changed dynamically by setting
the `cluster.info.update.interval` setting. This ClusterInfoService
can hopefully be used in the future to weight nodes for allocation
based on their disk usage, if desired.

The second main piece is the DiskThresholdDecider, which can disallow
a shard from being allocated to a node, or from remaining on the node
depending on configuration parameters. There are three main
configuration parameters for the DiskThresholdDecider:

`cluster.routing.allocation.disk.threshold_enabled` controls whether
the decider is enabled. It defaults to false (disabled). Note that the
decider is also disabled for clusters with only a single data node.

`cluster.routing.allocation.disk.watermark.low` controls the low
watermark for disk usage. It defaults to 0.70, meaning ES will not
allocate new shards to nodes once they have more than 70% disk
used. It can also be set to an absolute byte value (like 500mb) to
prevent ES from allocating shards if less than the configured amount
of space is available.

`cluster.routing.allocation.disk.watermark.high` controls the high
watermark. It defaults to 0.85, meaning ES will attempt to relocate
shards to another node if the node disk usage rises above 85%. It can
also be set to an absolute byte value (similar to the low watermark)
to relocate shards once less than the configured amount of space is
available on the node.

Closes #3480
2013-09-09 09:49:30 -06:00
Clinton Gormley 9e6d30a14a [DOCS] Changed the deprecation of custom_boost/score/filters_score
queries to 0.90.4
2013-09-05 12:14:10 +02:00
Clinton Gormley 2b3a762c27 [DOCS] Function score was added in 0.90.4 not 1.00.Beta 2013-09-05 11:25:06 +02:00
Clinton Gormley 8257aba166 [DOCS] Fixed fielddata regex syntax 2013-09-04 23:20:56 +02:00
Clinton Gormley 6d667e5d41 [DOCS] Missing sort values now works for all field types 2013-09-04 23:20:55 +02:00
Clinton Gormley 765bd026f5 [DOCS] Added function score query 2013-09-04 23:20:55 +02:00
Clinton Gormley aa59ef2e84 [DOCS] Added the human flag 2013-09-04 23:20:55 +02:00
Clinton Gormley 9d0dd545cb [DOCS] Tidied up the plugins page and added Graphite and Statsd 2013-09-04 23:20:55 +02:00
Clinton Gormley e1c6f45ff0 [DOCS] Added clarification about global scope in facets 2013-09-04 23:20:55 +02:00
Clinton Gormley 08f8e77b8f [DOCS] Added fuzzy options to completion suggester 2013-09-04 23:20:55 +02:00
Clinton Gormley 047c86e3b2 [DOCS] Added wildcard template matching 2013-09-04 23:20:55 +02:00
Clinton Gormley 9f5d0b6e89 [DOCS] Added a few clarifications to the docs from the issues list 2013-09-04 23:20:55 +02:00
Clinton Gormley 94be785726 [DOCS] Added multi-index open/close 2013-09-04 23:20:55 +02:00
Clinton Gormley 5b60506b2e [DOCS] Added highlighting to the phrase suggester 2013-09-04 23:20:54 +02:00
Clinton Gormley 53ad7330fc [DOCS] Added docs for term vectors 2013-09-04 23:20:54 +02:00
Clinton Gormley eac2b3a52e [DOCS] Fixed typo 2013-09-04 23:20:54 +02:00
Clinton Gormley 393c28bee4 [DOCS] Removed outdated new/deprecated version notices 2013-09-03 21:28:31 +02:00
Simon Willnauer eb2fed85f1 Add 'min_input_len' to completion suggester
Restrict the size of the input length to a reasonable size otherwise very
long strings can cause StackOverflowExceptions deep down in lucene land.
Yet, this is simply a saftly limit set to `50` UTF-16 codepoints by default.
This limit is only present at index time and not at query time. If prefix
completions > 50 UTF-16 codepoints are expected / desired this limit should be raised.
Critical string sizes are beyone the 1k UTF-16 Codepoints limit.

Closes #3596
2013-09-03 10:26:37 +02:00
Boaz Leskes e807c99f27 Fixed a typo in the config of light finnish stemmer (old last_finish is still supported for backward compatibility)
Closes #3594
2013-08-29 10:15:40 +02:00
Clinton Gormley 822043347e Migrated documentation into the main repo 2013-08-29 01:24:34 +02:00