This change enables bulk-merge for term vectors with index sort. The
algorithm used here is similar to the one that is used to merge stored
fields.
Relates #134
This change adds new IndexSearcher and Collector implementations to profile
search execution and break down the timings. The breakdown includes the total
time spent in each of the following categories along with the number of times
visited: create weight, build scorer, next doc, advance, score, match.
Co-authored-by: Julie Tibshirani <julietibs@gmail.com>
Before when creating a VectorWriter for merging, we would always load the
default implementation. So if the format was configured with parameters, they
were ignored.
This issue was caught by `TestKnnGraph#testMergeProducesSameGraph`.
Previously, the max connections and beam width parameters could be configured as
field type attributes. This PR moves them to be parameters on
Lucene90HnswVectorFormat, to avoid exposing details of the vector format
implementation in the API.
Boosts are ignored on inner span queries, and top-level boosts can
be applied by using a normal BoostQuery, so SpanBoostQuery
itself is redundant and trappy. This commit removes it entirely.
Before, rewriting could slightly change the scoring when weights were
specified. We now rewrite less aggressively to avoid changing the query's
behavior.
If we fail to delete files that belong to a commit point, then we will
expose that deleted commit in the next calls of IndexDeletionPolicy#onCommit.
I think we should never expose those deleted commit points as
some of their files might have been deleted already.
TestMatchesIterator lives in core/tests and does various sanity checks
on the matches returned by various queries, including Span queries.
The Span-specific tests cannot stay here once Spans have been moved
out of core. This commit pulls various helper methods from this class
into a base class in the test framework, so that we can move the
Spans tests into their own class and keep coverage once things have
been migrated.
We have a number of helper classes in o.a.l.search that aid the
implementation of two-phase iteration over disjunctions. These have
some Spans-specific code, which will stop compiling once Spans
are moved into the queries module. This commit removes the
Spans references from the main code and duplicates the helper
code within the Spans package.
ConjunctionDISI is really an internal implementation of DocIdSetIterator,
and would ideally be package-private. However, it is used in a few
other places:
* directly in ConjunctionSpans
* as a utility in the facet and join modules
This commit adds a public helper class ConjunctionUtils that allows easy
intersection of iterators for use by other modules. This means that
ConjunctionDISI itself can become package-private. It also removes
a reference to Spans from core classes, which will make it easier to
migrate Spans to the queries module. ConjuctionSpans and
ConjunctionIntervalIterator now use the public Utils class, and intervals
no longer need their own ConjunctionDISI implementation.
We need to update the reading logic of the backward codec in Lucene 9
for LUCENE-9827 and LUCENE-9935 as we have backported them to Lucene 8.
Relates apache/lucene-solr#2495
Relates apache/lucene-solr#2494
Dawid Weiss
Nhat Nguyen
Jack Conradson
Adrien Grand
Michael Sokolov
zacharymorn
Julie Tibshirani
Christine Poerschke
Alan Woodward
Greg Miller
Jan Høydahl
Chris Hostetter
Naoto MINAMI
Mike McCandless
Gautam Worah
Alexander Lukyanchikov
Tomoko Uchida
Houston Putman