From eb9f59f04196970a7eb4b408622b488841498a4e Mon Sep 17 00:00:00 2001 From: Erick Erickson Date: Thu, 27 Aug 2020 14:38:33 -0400 Subject: [PATCH] Adding forceMergeDeletePctAllowed and deletesPctAllowed to TieredMergePolicy documentation --- solr/solr-ref-guide/src/indexconfig-in-solrconfig.adoc | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/solr/solr-ref-guide/src/indexconfig-in-solrconfig.adoc b/solr/solr-ref-guide/src/indexconfig-in-solrconfig.adoc index 7759674aa96..8615a4b8c9c 100644 --- a/solr/solr-ref-guide/src/indexconfig-in-solrconfig.adoc +++ b/solr/solr-ref-guide/src/indexconfig-in-solrconfig.adoc @@ -82,6 +82,9 @@ Other policies available are the `LogByteSizeMergePolicy`, `LogDocMergePolicy`, 10 10 + 10.0 + 33.0 + ---- @@ -104,6 +107,13 @@ Choosing the best merge factors is generally a trade-off of indexing speed vs. s Conversely, keeping more segments can accelerate indexing, because merges happen less often, making an update is less likely to trigger a merge. But searches become more computationally expensive and will likely be slower, because search terms must be looked up in more index segments. Faster index updates also means shorter commit turnaround times, which means more timely search results. +=== Controlling Deleted Document Percentages + +When a document is deleted or updated, the document is marked as deleted but it not removed from the index until the segment is merged. There are two parameters that can can be adjusted when using the default TieredMergePolicy that influence the number of deleted documents in an index. + +* `forceMergeDeletesPctAllowed (default 10.0)`. When the external expungeDeletes command is issued, any segment that has more than this percent deleted documents will be merged into a new segment and the data associated with the deleted documents will be purged. A value of 0.0 will make expungeDeletes behave essentially identically to `optimize`. +* `deletesPctAllowed (default 33.0)`. During normal segment merging, a "best effort" is made to insure that the total percentage of deleted documents in the index is below this threshold. Valid settings are between 20% and 50%. 33% was chosen as the default because as this setting approaches 20%, considerable load is added to the system. + === Customizing Merge Policies If the configuration options for the built-in merge policies do not fully suit your use case, you can customize them: either by creating a custom merge policy factory that you specify in your configuration, or by configuring a {solr-javadocs}/solr-core/org/apache/solr/index/WrapperMergePolicyFactory.html[merge policy wrapper] which uses a `wrapped.prefix` configuration option to control how the factory it wraps will be configured: