Disable ConcurrentMergeScheduler's auto I/O throttling by default. (#13293)

This is motivated by the fact that merges can hardly steal all I/O resources
from searches on modern NVMe drives. Merges are still not allowed to use all
CPU since they have a budget for the number of threads which is a fraction of
the number of threads that the host can run.

Closes #13193

lucene/CHANGES.txt

@@ -177,6 +177,9 @@ Changes in Runtime Behavior
   `-Dorg.apache.lucene.store.defaultReadAdvice=normal`. This may be useful on systems
   with lots of RAM as this increases read-ahead. (Adrien Grand, Uwe Schindler)
 
+* GITHUB13293: Auto I/O throttling is now disabled by default on ConcurrentMergeScheduler.
+  (Adrien Grand)
+
 Changes in Backwards Compatibility Policy
 -----------------------------------------
 

lucene/MIGRATE.md

@@ -242,6 +242,12 @@ List<Object> results = searcher.search(query, new CustomCollectorManager());
 6. `ByteBuffersDataInput#size()`. Use `ByteBuffersDataInput#length()` instead
 7. `SortedSetDocValuesFacetField#label`. `FacetsConfig#pathToString(String[])` can be applied to path as a replacement if string path is desired.
 
+### Auto I/O throttling disabled by default in ConcurrentMergeScheduler (GITHUB#13293)
+
+ConcurrentMergeScheduler now disables auto I/O throttling by default. There is still some throttling
+happening at the CPU level, since ConcurrentMergeScheduler has a maximum number of threads it can
+use, which is only a fraction of the total number of threads of the host by default.
+
 ## Migration from Lucene 9.0 to Lucene 9.1
 
 ### Test framework package migration and module (LUCENE-10301)

lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java

@@ -108,7 +108,7 @@ public class ConcurrentMergeScheduler extends MergeScheduler {
   protected double targetMBPerSec = START_MB_PER_SEC;
 
   /** true if we should rate-limit writes for each merge */
-  private boolean doAutoIOThrottle = true;
+  private boolean doAutoIOThrottle = false;
 
   private double forceMergeMBPerSec = Double.POSITIVE_INFINITY;
 
@@ -202,7 +202,8 @@ public class ConcurrentMergeScheduler extends MergeScheduler {
 
   /**
    * Turn on dynamic IO throttling, to adaptively rate limit writes bytes/sec to the minimal rate
-   * necessary so merges do not fall behind. By default this is enabled.
+   * necessary so merges do not fall behind. By default this is disabled and writes are not
+   * rate-limited.
    */
   public synchronized void enableAutoIOThrottle() {
     doAutoIOThrottle = true;

lucene/core/src/test/org/apache/lucene/index/TestConcurrentMergeScheduler.java

@@ -900,10 +900,11 @@ public class TestConcurrentMergeScheduler extends LuceneTestCase {
 
   public void testAutoIOThrottleGetter() throws Exception {
     ConcurrentMergeScheduler cms = new ConcurrentMergeScheduler();
-    cms.disableAutoIOThrottle();
     assertFalse(cms.getAutoIOThrottle());
     cms.enableAutoIOThrottle();
     assertTrue(cms.getAutoIOThrottle());
+    cms.disableAutoIOThrottle();
+    assertFalse(cms.getAutoIOThrottle());
   }
 
   public void testNonSpinningDefaults() throws Exception {
@@ -944,6 +945,7 @@ public class TestConcurrentMergeScheduler extends LuceneTestCase {
             super.doStall();
           }
         };
+    cms.enableAutoIOThrottle();
     cms.setMaxMergesAndThreads(2, 1);
     iwc.setMergeScheduler(cms);
     iwc.setMaxBufferedDocs(2);