When the reader has no live docs, `KnnVectorQuery` can error out. This happens
because `IndexReader#numDocs` is 0, and we end up passing an illegal value of
`k = 0` to the search method.
This commit removes the problematic optimization in `KnnVectorQuery` and
replaces with a lower-level based on the total number of vectors in the segment.
When we load a query into the query cache we always calculate the count
of matching documents. This uses that count to power the new `O(1)`
`Weight#count` method.
This patch adds getPointValues to NumericLeafComparatorsimilar how it
has getNumericDocValues.
Numeric Sort optimization with points relies on the assumption that
points and doc values record the same information, as we substitute
iterator over doc_values with one over points.
If we override getNumericDocValues it almost certainly means that whatever
PointValues NumericComparator is going to look at shouldn't be used to
skip non-competitive documents. Returning null for pointValues in this
case will force comparator NOT to use sort optimization with points,
and continue with a traditional way of iterating over doc values.
LZ4 is interesting because it used to read data in little-endian order even
though Directory APIs were big endian. So most calls to LZ4 in backward-codecs
have been changed to change the endianness of the input/output.
While VectorSimilarityFunction#COSINE is helpful when you need to preserve the
original vectors, it is significantly slower than DOT_PRODUCT. This commit adds
javadocs to COSINE explaining that dot product is the fastest option.
* Java17 fixes
* Add to error message that the unexpected file is in lucene/ folder
* Fix gpg command utf-8 output
* Add --no-daemon to all gradle calls, and skip clean
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
Co-Authored-by: Tomoko Uchida <tomoko.uchida.1111@gmail.com>
We introduced invalid accesses for sorted set doc values in LUCENE-9613.
However, the issue has been unnoticed because the ordinals in doc values
tests aren't complex enough to use high packed bits, and the 3 padding
bytes make these invalid accesses perfectly fine. To reproduce this
issue, we need to use at least 20 bits per value for the ordinals.
* LUCENE-10185: pass --release 11 to ECJ linter, fix JDK 17 build
Otherwise, new java releases such as JDK 18, JDK 19, ... may have even
more new deprecations, the build shouldn't fail in such cases.
Remove -source/-target now that we pass --release
Fix casting so ECJ understands it and creates correct call signature (UweSays: "It's ok. I know why it happens, but it's a bug in ECJ. The type safety is checked by the invokeexact")
Co-authored-by: Uwe Schindler <uschindler@apache.org>
* LUCENE-9997 Revisit smoketester for 9.0 build
* Remove checkBrokenLinks
* Add back checkBrokenLinks
* Review feedback. Remove traces of solr-specific testNotice() method
Move backCompat test up to other "if isSrc" block
* Review feedback. Bring back the 'checkMaven()' method, as it checks lucene maven artifacts.
But since we dont have pom template files anymore, no need to compare with templates
* Review feedback. Fix script compatibility by comparing against X.Y instead of X.Y.Z
* Review feedback. Remove unnecessary if lucene test
Convert some ant commands to gradle
* Update MANIFEST tests to match the gradle-produced manifest
* LUCENE-10107 Read multi-line commit from Manifest
Backport from branch_8x
* Collapse for project in 'lucene' loops and methods taking 'project' as argument
Disable checkJavadocLinks, as this dependency no longer exists in 'scripts' folder
* Review feedback - fix more ant stuff, convert to gradle equivalent
* Review feedback: Refactor file open
* Comment out javadoc generation - was only used to check broken links?
* Fix charset of gpg console output to always be utf-8
Fix two more places to use with open()
* Accept 'LICENSE' without txt or md suffix in top-level
* Disable vector dictionary abuse exception if started with -Dsmoketester
* Reformat code
* Use -Dsmoketester flag when invoking IndexFiles
Mike McCandless
Dawid Weiss
Julie Tibshirani
Nik Everett
Mayya Sharipova
David Smiley
Adrien Grand
Nhat Nguyen
Julie Tibshirani
Jan Høydahl
Tomoko Uchida
Stefan Vodita
Robert Muir
Shintaro Murakami
Mike Drob
Chris Hostetter
Tommaso Teofili