Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Apache Lucene open-source search software
32,851 Commits 29 Branches 469 Tags 816 MiB
Java 97.7%
HTML 1%
Python 0.9%
Lex 0.3%
Go to file
msfroh a1791e7714 LUCENE-8962: Add ability to selectively merge on commit (#1155) 
* LUCENE-8962: Add ability to selectively merge on commit

This adds a new "findCommitMerges" method to MergePolicy, which can
specify merges to be executed before the
IndexWriter.prepareCommitInternal method returns.

If we have many index writer threads, they will flush their DWPT buffers
on commit, resulting in many small segments, which can be merged before
the commit returns.

* Add missing Javadoc

* Fix incorrect comment

* Refactoring and fix intermittent test failure

1. Made some changes to the callback to update toCommit, leveraging
SegmentInfos.applyMergeChanges.
2. I realized that we'll never end up with 0 registered merges, because
we throw an exception if we fail to register a merge.
3. Moved the IndexWriterEvents.beginMergeOnCommit notification to before
we call MergeScheduler.merge, since we may not be merging on another
thread.
4. There was an intermittent test failure due to randomness in the time
it takes for merges to complete. Before doing the final commit, we wait
for pending merges to finish. We may still end up abandoning the final
merge, but we can detect that and assert that either the merge was
abandoned (and we have > 1 segment) or we did merge down to 1 segment.

* Fix typo

* Fix/improve comments based on PR feedback

* More comment improvements from PR feedback

* Rename method and add new MergeTrigger

1. Renamed findCommitMerges -> findFullFlushMerges.
2. Added MergeTrigger.COMMIT, passed to findFullFlushMerges and to
   MergeScheduler when merging on commit.

* Update renamed method name in strings and comments
 		2020-03-02 18:09:03 -05:00
dev-docs SOLR-12930: Create developer docs directories in source repo (#1164) 2020-01-23 14:11:31 -06:00
dev-tools SOLR-14149: CHANGES.txt Remove off-topic stuff 2020-02-08 23:04:41 -05:00
lucene LUCENE-8962: Add ability to selectively merge on commit (#1155) 2020-03-02 18:09:03 -05:00
solr SOLR-14241: fix typos & incorrect example param 2020-03-02 16:53:46 -06:00
.gitignore LUCENE-9180: dos2unix files that don't need dos line endings. gitignore gradle-specific stuff that shows up modified if you switch branches, no gradle here. 2020-01-27 11:31:59 -05:00
.hgignore LUCENE-2792: add FST impl 2010-12-12 15:36:08 +00:00
README.md SOLR-13640: update README 2019-07-20 18:35:13 +05:30
build.xml LUCENE-9230: explicitly call python version we want from builds 2020-02-18 19:04:37 -05:00

README.md

Apache Lucene and Solr

Apache Lucene is a high-performance, full featured text search engine library written in Java.

Apache Solr is an enterprise search platform written using Apache Lucene. Major features include full-text search, index replication and sharding, and result faceting and highlighting.

Online Documentation

This README file only contains basic setup instructions. For more comprehensive documentation, visit:

Building Lucene/Solr

(You do not need to do this if you downloaded a pre-built package.)

Lucene and Solr are built using Apache Ant. To build Lucene and Solr, run:

ant compile

If you see an error about Ivy missing while invoking Ant (e.g., .ant/lib does not exist), run ant ivy-bootstrap and retry.

Sometimes you may face issues with Ivy (e.g., an incompletely downloaded artifact). Cleaning up the Ivy cache and retrying is a workaround for most of such issues:

rm -rf ~/.ivy2/cache

The Solr server can then be packaged and prepared for startup by running the following command from the solr/ directory:

ant server

Running Solr

After building Solr, the server can be started using the bin/solr control scripts. Solr can be run in either standalone or distributed (SolrCloud mode).

To run Solr in standalone mode, run the following command from the solr/ directory:

bin/solr start

To run Solr in SolrCloud mode, run the following command from the solr/ directory:

bin/solr start -c

The bin/solr control script allows heavy modification of the started Solr. Common options are described in some detail in solr/README.txt. For an exhaustive treatment of options, run bin/solr start -h from the solr/ directory.

Development/IDEs

Ant can be used to generate project files compatible with most common IDEs. Run the ant command corresponding to your IDE of choice before attempting to import Lucene/Solr.

  • Eclipse - ant eclipse (See this for details)
  • IntelliJ - ant idea (See this for details)
  • Netbeans - ant netbeans (See this for details)

Running Tests

The standard test suite can be run with the command:

ant test

Like Solr itself, the test-running can be customized or tailored in a number or ways. For an exhaustive discussion of the options available, run:

ant test-help

Contributing

Please review the Contributing to Solr Guide for information on contributing.

Discussion and Support