* Use merge policy and merge scheduler to run addIndexes merges
* wrapped reader does not see deletes - debug
* Partially fixed tests in TestAddIndexes
* Use writer object to invoke addIndexes merge
* Use merge object info
* Add javadocs for new methods
* TestAddIndexes passing
* verify field info schemas upfront from incoming readers
* rename flag to track pooled readers
* Keep addIndexes API transactional
* Maintain transactionality - register segments with iw after all merges complete
* fix checkstyle
* PR comments
* Fix pendingDocs - numDocs mismatch bug
* Tests with 1-1 merges and partial merge failures
* variable renaming and better comments
* add test for partial merge failures. change tests to use 1-1 findmerges
* abort pending merges gracefully
* test null and empty merge specs
* test interim files are deleted
* test with empty readers
* test cascading merges triggered
* remove nocommits
* gradle check errors
* remove unused line
* remove printf
* spotless apply
* update TestIndexWriterOnDiskFull to accept mergeException from failing addIndexes calls
* return singleton reader mergespec in NoMergePolicy
* rethrow exceptions seen in merge threads on failure
* spotless apply
* update test to new exception type thrown
* spotlessApply
* test for maxDoc limit in IndexWriter
* spotlessApply
* Use DocValuesIterator instead of DocValuesFieldExistsQuery for counting soft deletes
* spotless apply
* change exception message for closed IW
* remove non-essential comments
* update api doc string
* doc string update
* spotless
* Changes file entry
* simplify findMerges API, add 1-1 merges to MockRandomMergePolicy
* update merge policies to new api
* remove unused imports
* spotless apply
* move changes entry to end of list
* fix testAddIndicesWithSoftDeletes
* test with 1-1 merge policy always enabled
* please spotcheck
* tidy
* test - never use 1-1 merge policy
* use 1-1 merge policy randomly
* Remove concurrent addIndexes findMerges from MockRandomMergePolicy
* Bug Fix: RuntimeException in addIndexes
Aborted pending merges were slipping through the merge exception check in
API, and getting caught later in the RuntimeException check.
* tidy
* Rebase on main. Move changes to 10.0
* Synchronize IW.AddIndexesMergeSource on outer class IW object
* tidy
I noticed some minor bugs in the original PR #927 that this PR should fix:
- When a timeout is set, we would no longer catch
`CollectionTerminatedException`.
- I added randomization to `LuceneTestCase` to randomly set a timeout, it
would have caught the above bug.
- Fixed visibility of `TimeLimitingBulkScorer`.
Before we were checking the number of vectors in the segment against the total
number of documents in IndexReader. This meant FieldExistsQuery would not
rewrite to MatchAllDocsQuery when there were multiple segments.
Instead of collecting hit-by-hit using a `LeafCollector`, we break down the
search by instantiating a weight, creating scorers, and checking the underlying
iterator. If it is backed by a `BitSet`, we directly update the reference (as
we won't be editing the `Bits`). Else we can create a new `BitSet` from the
iterator using `BitSet.of`.
The test used to leave hanging threads behind following a failure. Also one method was executing two different tests. I split the existing method into two and I am now leveraging setup and teardown to properly close all the resources both when the tests succeed as well as whey they fail.
As suggested by @zhaih on #950, we could support more cases in
`BooleanWeight#count`. This PR adds support for these cases specifically:
- Pure disjunctions where only one clause has a non-zero count.
- Pure disjunctions where one clause matches all docs.
- Negations where positive clauses match all docs (pure negation).
- Negations where positive clauses match no docs.
- Negations where negative clauses match no docs.
- Negations where negative clauses match all docs.
The HNSW graph search does not consider that visitedLimit may be reached in the
upper levels of graph search itself
This occurs when the pre-filter is too restrictive (and its count sets the
visitedLimit). So instead of switching over to exactSearch, it tries to pop
from an empty heap and throws an error.
We can check if results are incomplete after searching in upper levels, and
break out accordingly. This way it won't throw heap errors, and gracefully
switch to exactSearch instead
Vigya Sharma
Peter Gromov
Adrien Grand
zacharymorn
Julie Tibshirani
Uwe Schindler
Greg Miller
Michael Sokolov
Deepika0510
Tomoko Uchida
Dawid Weiss
Alessandro Benedetti
Lu Xugang
Kaival Parikh
Shai Erera
Marc D'mello
Christine Poerschke
Luca Cavanna
tang donghai
Gautam Worah
Mike Drob
Yuting Gan
Mike Drob
JoeHF