Benjamin Trent e3a34bfe56
Add new parallel merge task executor for parallel actions within a single merge action (#13124)
This commit adds a new interface to all MergeScheduler classes that allows the scheduler to provide an Executor for intra-merge parallelism. The first sub-class to satisfy this new interface is the ConcurrentMergeScheduler (CMS). In particular, the CMS will limit the amount of parallelism based on the current number of mergeThreads and the configured maxThread options. 

Parallelism is now added by default to the SegmentMerger where each merging task is executed in parallel with the others. 

Additionally, the Lucene99 HNSW codec will use the provided MergeScheduler executor if no executor is provided directly.

Relates to: https://github.com/apache/lucene/issues/12740
Relates to: https://github.com/apache/lucene/issues/9626
2024-03-14 15:58:41 -04:00

Apache Lucene

Lucene Logo

Apache Lucene is a high-performance, full-featured text search engine library written in Java.

Build Status

Online Documentation

This README file only contains basic setup instructions. For more comprehensive documentation, visit:

Building

Basic steps:

  1. Install OpenJDK 21.
  2. Clone Lucene's git repository (or download the source distribution).
  3. Run gradle launcher script (gradlew).

We'll assume that you know how to get and set up the JDK - if you don't, then we suggest starting at https://jdk.java.net/ and learning more about Java, before returning to this README.

See Contributing Guide for details.

Contributing

Bug fixes, improvements and new features are always welcome! Please review the Contributing to Lucene Guide for information on contributing.

Discussion and Support

Description
Apache Lucene open-source search software
Readme 644 MiB
Languages
Java 97.7%
HTML 1%
Python 0.9%
Lex 0.3%