mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-17 18:35:25 +00:00
992747a159
Today, even though our merge policy doesn't return new merge specs on SEGMENT_FLUSH, merge on the scheduler is still called on flush time, and can cause merges to stall indexing during merges. Both for the concurrent merge scheduler (the default) and the serial merge scheduler. This behavior become worse when throttling kicks in (today at 20mb per sec). In order to solve it (outside of Lucene for now), we wrap the merge scheduler with an EnableMergeScheduler, where, on the thread level, using a thread local, the call to merge can be enabled/disabled. A Merges helper class is added where all explicit merges operations should go through. If the scheduler is the enabled one, it will enable merges before calling the relevant explicit method call. In order to make sure Merges is the only class that calls the explicit merge calls, the IW variant of them is added to the forbidden APIs list. closes #5319
33 lines
1.9 KiB
Plaintext
33 lines
1.9 KiB
Plaintext
@defaultMessage spawns threads with vague names; use a custom thread factory and name threads so that you can tell (by its name) which executor it is associated with
|
|
|
|
java.util.concurrent.Executors#newFixedThreadPool(int)
|
|
java.util.concurrent.Executors#newSingleThreadExecutor()
|
|
java.util.concurrent.Executors#newCachedThreadPool()
|
|
java.util.concurrent.Executors#newSingleThreadScheduledExecutor()
|
|
java.util.concurrent.Executors#newScheduledThreadPool(int)
|
|
java.util.concurrent.Executors#defaultThreadFactory()
|
|
java.util.concurrent.Executors#privilegedThreadFactory()
|
|
|
|
java.lang.Character#codePointBefore(char[],int) @ Implicit start offset is error-prone when the char[] is a buffer and the first chars are random chars
|
|
java.lang.Character#codePointAt(char[],int) @ Implicit end offset is error-prone when the char[] is a buffer and the last chars are random chars
|
|
|
|
@defaultMessage Collections.sort dumps data into an array, sorts the array and reinserts data into the list, one should rather use Lucene's CollectionUtil sort methods which sort in place
|
|
|
|
java.util.Collections#sort(java.util.List)
|
|
java.util.Collections#sort(java.util.List,java.util.Comparator)
|
|
|
|
java.io.StringReader#<init>(java.lang.String) @ Use FastStringReader instead
|
|
|
|
org.apache.lucene.util.RamUsageEstimator#sizeOf(java.lang.Object) @ This can be a perfromance trap
|
|
|
|
@defaultMessage Reference management is tricky, leave it to SearcherManager
|
|
org.apache.lucene.index.IndexReader#decRef()
|
|
org.apache.lucene.index.IndexReader#incRef()
|
|
org.apache.lucene.index.IndexReader#tryIncRef()
|
|
|
|
org.apache.lucene.index.IndexWriter#maybeMerge() @ use Merges#maybeMerge
|
|
org.apache.lucene.index.IndexWriter#forceMerge(int) @ use Merges#forceMerge
|
|
org.apache.lucene.index.IndexWriter#forceMerge(int,boolean) @ use Merges#forceMerge
|
|
org.apache.lucene.index.IndexWriter#forceMergeDeletes() @ use Merges#forceMergeDeletes
|
|
org.apache.lucene.index.IndexWriter#forceMergeDeletes(boolean) @ use Merges#forceMergeDeletes
|