Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
37,291 Commits 29 Branches 469 Tags 740 MiB
Commit Graph

37291 Commits

Author SHA1 Message Date
Adrien Grand 61f322905a
Change BP reordering logic to help support document blocks later on. (#13123) 
The current logic for reordering splits a slice of doc IDs into a left side and
a right side, and for each document it computes the expected gain of moving to
the other side. Then it swaps documents from both sides as long as the sum of
the gain of moving the left doc to the right and the right doc to the left is
positive.

This works well, but I would like to extend BP reordering to also work with
blocks, and the swapping logic is challenging to modify as two parent documents
may have different numbers of children.

One of the follow-up papers on BP suggested to use a different logic, where one
would compute a bias for all documents that is negative when a document is
attracted to the left and positive otherwise. Then we only have to partition doc
IDs around the mid point, e.g. with quickselect.

A benefit of this change is that it will make it easier to generalize BP
reordering to indexes that have blocks, e.g. by using a stable sort on biases.
 2024-02-23 09:09:19 +01:00
Chris Hegarty 17cbedccfc
FieldInfosFormat translation should be independent of VectorSimilartyFunction enum (#13119) 
This commit updates the FieldInfosFormat translation of vector similarity functions to be independent of the VectorSimilartyFunction enum.

The VectorSimilartyFunction enum lives outside of the codec format, and the format should not inadvertently depend upon the declaration order or values in VectorSimilartyFunction. The format should be in charge of the translation of similarity function to format ordinal (and visa versa). In reality, and for now, the translation remains the same as the declaration order, but this may not be the case in the future.
 2024-02-22 17:22:20 +00:00
Zhang Chao 6732d2b2fa
fix (#13122) 2024-02-22 14:11:07 +08:00
Adrien Grand 568c8eba97 Use NIO2 APIs. 2024-02-20 22:37:18 +01:00
Adrien Grand 75b26b1bee Bump min version with parent field. 2024-02-20 22:29:11 +01:00
Adrien Grand dce66f1e3e Add back-compat indices for 9.10.0 2024-02-20 22:16:56 +01:00
Adrien Grand 13d561af1d Fix bw index generation logic. 2024-02-20 22:14:06 +01:00
Adrien Grand 47b02dba1e DOAP changes for release 9.10.0 2024-02-20 18:17:14 +01:00
Prabhat ff59150934
Added queued flush check optimization to changes.txt (#13118) 
Co-authored-by: Prabhat Sharma <ptsharma@amazon.com>
 2024-02-20 17:21:05 +01:00
Prabhat 8a73cdef8c
Short circuit queued flush check when flush on update is disabled (#13115) 
Co-authored-by: Prabhat Sharma <ptsharma@amazon.com>
 2024-02-19 09:21:38 -05:00
ChrisHegarty c7b05f56bd post merge fix 2024-02-19 11:46:17 +00:00
ChrisHegarty 07f4b5b19f Merge branch 'main' into java_21 2024-02-19 11:43:46 +00:00
Zhang Chao 55df3e02f3
Fix too many open files Exception for some tests (#13035) 
* init

* fix review

* fix review

* iter
 2024-02-19 05:35:24 -05:00
Dmitry Cherniachenko 9a88d8ad25
Remove redundant field initializers (#13060) 
Make protected fields in final classes private.
 2024-02-19 09:52:43 +00:00
Benjamin Trent 39f681812b
Adding a getter to FeatureField to allow value access (#13111) 
Getting the access of a FeatureField#value is useful for deduplicating. If you have a sparse vector model and you want to handle multiple inputs from them, you want flexibility in how you de-duplicate the feature dimensions.
 2024-02-17 09:03:08 -05:00
Benjamin Trent af0d0edf5e
Fix test failure in #13057 (#13102) 
The original tests assume particular document orders & scores. To make the test more resilient to random flushes & merges, I adjusted the assertion conditions. Particularly, we verify matching ids -> score instead of relying on docIds.

closes: https://github.com/apache/lucene/issues/13057
 2024-02-16 09:33:34 -05:00
Dawid Weiss cc25fd9deb React to mod-analysis-common workflow path (regression after workflow renamed). #13110 2024-02-16 12:40:49 +01:00
Dawid Weiss aa7d035ac2
github workflow cleanup and minor improvements (#13110) 2024-02-16 12:38:21 +01:00
Dmitry Cherniachenko 9206bdeb06
Move `brToString(BytesRef)` to `ToStringUtils` (#13068) 2024-02-15 18:16:44 +01:00
Zhang Chao b16d7117a3
Fix NPE for TestRandomChains.testRandomChainsWithLargeStrings (#13104) 2024-02-15 16:20:53 +01:00
Dmitry Cherniachenko f24b1de351
Cleanup TokenizedPhraseQueryNode code (#13041) 2024-02-15 10:04:58 +00:00
Simon Willnauer c9e4434cc3 Fix addBackcompatIndexes.py to properly generate missing versions (#13095) 
In #13046 several changes broke the addBackcompatIndexes.py script
to properly add and test the unreleased version. This updates the
script to again properly add the new version.

Closes #13094

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
 2024-02-14 21:58:30 +01:00
Dawid Weiss 444d816622 Moved changes entry to 9.11 #13092 2024-02-14 11:32:55 +01:00
Dmitry Cherniachenko 3768b2e1e0
Make `static final Map` constants immutable (#13092) 2024-02-13 19:02:55 +01:00
Dawid Weiss a270acae01
This reverts the addition of spotless:on/off regions and shows just one possible alternative that is formatter fool-proof. (#13098) 2024-02-13 19:00:11 +01:00
Uwe Schindler 2639aea5dd
Change the links to Lucene's homepage in the DOAP file to https:// (#13097) 2024-02-13 14:19:33 +01:00
Simon Willnauer c42380cbd0 Enable parent field in sorted bwc tests (#13067) 
This enables the optional parent field in BWC tests from 9.10 on.
This will need to be forward ported to main branch where the
parent field is required to these tests since they add document blocks
during tests.
 2024-02-13 12:00:29 +01:00
Adrien Grand 6f026d7c36 Fix logic to always exclude latest minor from previous major. 2024-02-13 10:59:16 +01:00
Adrien Grand d7e0a7f0e4
Only track released versions in `oldVersions`. (#13096) 2024-02-13 10:48:30 +01:00
Zhang Chao f6e065dc31
Fix test failure for TestDocumentsWriterDeleteQueue.testUpdateDeleteSlices (#13089) 2024-02-13 10:10:44 +08:00
Adrien Grand 027240f522 Tidy after version bump. 2024-02-12 17:01:47 +01:00
Adrien Grand 314c553bdc Add next major version 9.11.0 2024-02-12 14:36:17 +01:00
Christine Poerschke 8bf10130e9 lucene/MIGRATE.md update: remove empty 9.9 to 9.10 section 2024-02-12 10:17:41 +00:00
Dawid Weiss 7d99e6aafb Move changes and migration entries to Lucene 10.x since it seems there'll be no way to apply this patch in 9x? #12674 2024-02-11 11:14:58 +01:00
Uwe Schindler 7e921e4ed9 fix typo in CHANGES.txt 2024-02-09 23:15:05 +01:00
Uwe Schindler 178f5a7a7e
Enable MemorySegment in MMapDirectory for Java 22+ and Vectorization (incubation) for exact Java 22 (#12706) 2024-02-09 23:02:42 +01:00
Robert Muir d7a16dc10a
Merge branch 'main' into java_21 2024-02-09 15:22:45 -05:00
Robert Muir ec04490bd2
java 17 -> java 21 2024-02-09 15:18:30 -05:00
Robert Muir 23ecac438a
java 17 -> java 21 2024-02-09 15:17:46 -05:00
Robert Muir fefde0f721
java 17 -> java 21 2024-02-09 15:16:41 -05:00
Robert Muir 1f9545e830
remove java < 21 2024-02-09 15:15:52 -05:00
Robert Muir 784c331b68
java 17 -> java 21 2024-02-09 15:14:55 -05:00
Robert Muir a221b1f677
java 17 -> java 21 2024-02-09 15:14:20 -05:00
Robert Muir d75b869d21
java 17 -> java 21 2024-02-09 15:13:37 -05:00
Zhang Chao a1faa6398b
Add necessary assertion in CheckHits#doCheckMaxScores (#13088) 2024-02-09 18:28:18 +01:00
Christine Poerschke 563fafd8ac
upgrade to OpenNLP 2.3.2 (#12674) 2024-02-09 11:21:41 +00:00
Houston Putman 521c26e4fb Add back-compat indices for 8.11.3 2024-02-08 19:25:40 -06:00
Houston Putman efdbce412a Add bugfix version 8.11.3 2024-02-08 18:16:59 -06:00
Houston Putman 01ed004e94
Fix positioning of 8.11.3 DOAP RDF entry 2024-02-08 18:49:11 -05:00
Houston Putman 06101aa85d
Add 8.11.3 release to DOAP RDF file 2024-02-08 18:43:48 -05:00