From 870346070e3b3ddc483263df8d369fff111fdac7 Mon Sep 17 00:00:00 2001 From: Clinton Gormley Date: Tue, 15 Oct 2013 13:30:56 +0200 Subject: [PATCH] [DOCS] Added compound_on_flush docs and updated compound_format docs to include note about accepting a float --- docs/reference/index-modules.asciidoc | 30 +++++--- docs/reference/index-modules/merge.asciidoc | 71 +++++++++++-------- .../indices/update-settings.asciidoc | 25 ++++--- 3 files changed, 75 insertions(+), 51 deletions(-) diff --git a/docs/reference/index-modules.asciidoc b/docs/reference/index-modules.asciidoc index 57042a7bd89..e996820c8f2 100644 --- a/docs/reference/index-modules.asciidoc +++ b/docs/reference/index-modules.asciidoc @@ -15,16 +15,28 @@ all the relevant modules settings can be provided when creating an index There are specific index level settings that are not associated with any specific module. These include: -`index.compound_format`:: +[[index-compound-format]]`index.compound_format`:: - Should the compound file format be used (boolean - setting). If not set, controlled by the actually store used, this is - because the compound format was created to reduce the number of open - file handles when using file based storage. By default, it is set to - `false` for better performance (really applicable for file system based - index storage), but, requires adapting the max open file handles. + Should the compound file format be used (boolean setting). + The compound format was created to reduce the number of open + file handles when using file based storage. However, by default it is set + to `false` as the non-compound format gives better performance. It is important + that OS is configured to give Elasticsearch ``enough'' file handles. + See <>. ++ +Alternatively, `compound_format` can be set to a number between `0` and +`1`, where `0` means `false`, `1` means `true` and a number inbetween +represents a percentage: if the merged segment is less than this +percentage of the total index, then it is written in compound format, +otherwise it is written in non-compound format. added[0.90.2] -`index.term_index_interval`:: +[[index-compound-on-flush]]`index.compound_on_flush`:: + + Should a new segment (create by indexing, not by merging) be written + in compound format or non-compound format? Defaults to `true`. + This is a dynamic setting. + +`index.term_index_interval`:: Set the interval between indexed terms. Large values cause less memory to be used by a reader / searcher, but @@ -32,7 +44,7 @@ specific module. These include: by a reader / searcher, and speed random-access to terms. Defaults to `128`. -`index.term_index_divisor`:: +`index.term_index_divisor`:: Subsamples which indexed terms are loaded into RAM. This has the same effect as `index.term_index_interval` except that setting must be done at indexing time while this setting can be set diff --git a/docs/reference/index-modules/merge.asciidoc b/docs/reference/index-modules/merge.asciidoc index 2ca29fb4e50..61b29f11b27 100644 --- a/docs/reference/index-modules/merge.asciidoc +++ b/docs/reference/index-modules/merge.asciidoc @@ -36,44 +36,53 @@ over-merge (i.e., cascade merges). This policy has the following settings: -[cols="<,<",options="header",] -|======================================================================= -|Setting |Description -|`index.merge.policy.expunge_deletes_allowed` |When expungeDeletes is -called, we only merge away a segment if its delete percentage is over -this threshold. Default is `10`. +`index.merge.policy.expunge_deletes_allowed`:: -|`index.merge.policy.floor_segment` |Segments smaller than this are -"rounded up" to this size, i.e. treated as equal (floor) size for merge -selection. This is to prevent frequent flushing of tiny segments from -allowing a long tail in the index. Default is `2mb`. + When expungeDeletes is called, we only merge away a segment if its delete + percentage is over this threshold. Default is `10`. -|`index.merge.policy.max_merge_at_once` |Maximum number of segments to -be merged at a time during "normal" merging. Default is `10`. +`index.merge.policy.floor_segment`:: -|`index.merge.policy.max_merge_at_once_explicit` |Maximum number of -segments to be merged at a time, during optimize or expungeDeletes. -Default is `30`. + Segments smaller than this are "rounded up" to this size, i.e. treated as + equal (floor) size for merge selection. This is to prevent frequent + flushing of tiny segments from allowing a long tail in the index. Default + is `2mb`. -|`index.merge.policy.max_merged_segment` |Maximum sized segment to -produce during normal merging (not explicit optimize). This setting is -approximate: the estimate of the merged segment size is made by summing -sizes of to-be-merged segments (compensating for percent deleted docs). -Default is `5gb`. +`index.merge.policy.max_merge_at_once`:: -|`index.merge.policy.segments_per_tier` |Sets the allowed number of -segments per tier. Smaller values mean more merging but fewer segments. -Default is `10`. Note, this value needs to be >= then the -`max_merge_at_once_` otherwise you'll force too many merges to occur. + Maximum number of segments to be merged at a time during "normal" merging. + Default is `10`. -|`index.reclaim_deletes_weight` |Controls how aggressively merges that -reclaim more deletions are favored. Higher values favor selecting merges -that reclaim deletions. A value of `0.0` means deletions don't impact -merge selection. Defaults to `2.0`. +`index.merge.policy.max_merge_at_once_explicit`:: -|`index.compound_format` |Should the index be stored in compound format -or not. Defaults to `false`. -|======================================================================= + Maximum number of segments to be merged at a time, during optimize or + expungeDeletes. Default is `30`. + +`index.merge.policy.max_merged_segment`:: + + Maximum sized segment to produce during normal merging (not explicit + optimize). This setting is approximate: the estimate of the merged segment + size is made by summing sizes of to-be-merged segments (compensating for + percent deleted docs). Default is `5gb`. + +`index.merge.policy.segments_per_tier`:: + + Sets the allowed number of segments per tier. Smaller values mean more + merging but fewer segments. Default is `10`. Note, this value needs to be + >= then the `max_merge_at_once_` otherwise you'll force too many merges to + occur. + +`index.reclaim_deletes_weight`:: + + Controls how aggressively merges that reclaim more deletions are favored. + Higher values favor selecting merges that reclaim deletions. A value of + `0.0` means deletions don't impact merge selection. Defaults to `2.0`. + +`index.compound_format`:: + + Should the index be stored in compound format or not. Defaults to `false`. + See <> in + <>. For normal merging, this policy first computes a "budget" of how many segments are allowed by be in the index. If the index is over-budget, diff --git a/docs/reference/indices/update-settings.asciidoc b/docs/reference/indices/update-settings.asciidoc index 14a89aac746..2aed34dbc90 100644 --- a/docs/reference/indices/update-settings.asciidoc +++ b/docs/reference/indices/update-settings.asciidoc @@ -31,8 +31,6 @@ curl -XPUT 'localhost:9200/my_index/_settings' -d ' Below is the list of settings that can be changed using the update settings API: -[horizontal] - `index.number_of_replicas`:: The number of replicas each shard has. @@ -40,7 +38,7 @@ settings API: Set to an actual value (like `0-all`) or `false` to disable it. `index.blocks.read_only`:: - Set to `true` to have the index read only, `false` to allow writes + Set to `true` to have the index read only, `false` to allow writes and metadata changes. `index.blocks.read`:: @@ -80,23 +78,23 @@ settings API: When to flush based on a period of not flushing. `index.translog.disable_flush`:: - Disables flushing. Note, should be set for a short + Disables flushing. Note, should be set for a short interval and then enabled. `index.cache.filter.max_size`:: - The maximum size of filter cache (per segment in shard). + The maximum size of filter cache (per segment in shard). Set to `-1` to disable. `index.cache.filter.expire`:: - The expire after access time for filter cache. + The expire after access time for filter cache. Set to `-1` to disable. `index.gateway.snapshot_interval`:: - The gateway snapshot interval (only applies to shared gateways). + The gateway snapshot interval (only applies to shared gateways). Defaults to 10s. <>:: - All the settings for the merge policy currently configured. + All the settings for the merge policy currently configured. A different merge policy can't be set. `index.routing.allocation.include.*`:: @@ -125,7 +123,7 @@ settings API: * `quorum` (default) * `quorum-1` (or `half`) * `full` - * full-1`. + * full-1`. * Number values are also supported, e.g. `1`. `index.gc_deletes`:: @@ -140,9 +138,14 @@ settings API: Either `simple` or `buffered` (default). `index.compound_format`:: - See <> + See <> in + <>. -|<>:: +`index.compound_on_flush`:: + See <> in + <>. + +<>:: All the settings for slow log. `index.warmer.enabled`::