Commit Graph

36420 Commits

Author SHA1 Message Date
Karl David Wright fbdb655221 Add node structures and fast operations for them. 2022-11-19 17:56:08 -05:00
Robert Muir 62f2b42502
Prevent TestStressIndexing from taking minutes for normal non-NIGHTLY runs (#11953)
This test intentionally does a ton of filesystem operations: currently
about 20% of the time you can get really unlucky and get virus checker
simulated, against a real filesystem, which makes things really slow.

Instead use a ByteBuffersDirectory for local runs so that it doesn't
take minutes. The test can still be pretty slow even with this
implementation, so tone down the runtime so that it takes ~ 1.5s
locally.
2022-11-19 18:06:52 -05:00
Dawid Weiss 3f6410b738
Implement source code regeneration for test-framework perl scripts (#11952) 2022-11-19 23:40:45 +01:00
Dawid Weiss e78210b7f0
Add self-contained artifact upload script for apache nexus (#11329) (#11947) 2022-11-18 18:58:06 +01:00
Karl David Wright 718ae33e21 Merge branch 'main' of https://gitbox.apache.org/repos/asf/lucene into main 2022-11-18 02:04:54 -05:00
Dawid Weiss 82e57d1d6b Remove obsolete badapples tests switch (#468) 2022-11-18 16:57:24 +01:00
Robert Muir 99aa6dd0eb
Lower gradle heap: 3GB is unnecessary (#11936) 2022-11-18 10:55:32 -05:00
Dawid Weiss 2f21a866c1
Add star import check/validation (#11949)
* Remove some old cruft that only slows down checks. Add star import check
* Expand wildcard imports to comply with the rule.
2022-11-18 16:42:59 +01:00
Karl David Wright 492a25a2a2 Fix formatting 2022-11-18 01:41:53 -05:00
Karl David Wright f3b4b057fa Refactor, in preparation for using b-trees to enhance performance. 2022-11-18 00:32:52 -05:00
Jack Conradson a18b62ded4
Decrease test time for TestManyKnnDocs.testLargeSegment (#11945)
* Improve speed of TestManyKnnDocs
2022-11-16 23:52:32 -05:00
Karl David Wright b6ebfd1861 Prevent NPEs while still handling the polar case for horizontal planes right off the pole 2022-11-16 11:03:24 -05:00
Mao Suhan 3c5bcb383b
fix bug of incorrect cost after upgradeToBitSet in DocIdSetBuilder class (#11939) 2022-11-16 17:04:15 +01:00
Robert Muir c5da727493
fix overflows in compound assignments (#11938)
* Count points as longs.

* Simplify KnnVectorsWriter.

Co-authored-by: Adrien Grand <jpountz@gmail.com>
2022-11-16 10:53:34 -05:00
Dawid Weiss a5e2525946 An alternative workaround for gradle -J javac arg passing bug (#11937) 2022-11-16 09:46:24 +01:00
Robert Muir b292151f21
reland: 11925 javac options (#11937)
* pass jvm args to javac  #11925

* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging

* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

* handle the different possible alt toolchain cases

The difference is invoking 'java' versus invoking 'javac', so the args must be fed differently.

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-11-15 19:28:52 -05:00
Robert Muir 0753f48a80
Revert "speed up javac and error-prone tasks by using less resources (#11927)"
This reverts commit deab699188.

Having a shitty day: git lost changes :(
2022-11-15 19:08:43 -05:00
Robert Muir deab699188
speed up javac and error-prone tasks by using less resources (#11927)
* pass jvm args to javac  #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

Co-authored-by: Dawid Weiss <dweiss@apache.org>
2022-11-15 19:04:17 -05:00
Robert Muir a02db83235
Revert "speed up javac and error-prone tasks by using less resources (#11927)"
This reverts commit b9c51d9a59.
2022-11-15 18:19:09 -05:00
Dawid Weiss b9c51d9a59
speed up javac and error-prone tasks by using less resources (#11927)
* pass jvm args to javac  #11925
* Update net.ltgt.errorprone to the latest version so that jvm args are not overwritten, add -XX:+PrintFlagsFinal for debugging
* speed up the pure javac case too

It does not help to fork additional VMs (although error-prone will do
this since it messes with bootstrap classpath), so we avoid forking.

Instead it is best to tune org.gradle.jvmargs.

* use the flags consistently everywhere (tests and doc)

Co-authored-by: Robert Muir <rmuir@apache.org>
2022-11-15 16:06:38 -05:00
Michael McCandless 02528c6757
#10878: add some test verbosity on failure (#11935) 2022-11-15 14:20:04 -05:00
Adrien Grand e2e14df5ac Add changes entry for #11930. 2022-11-15 15:31:47 +01:00
Adrien Grand 729dc2bb82
Introduce IOContext.LOAD (#11930)
The default codec has a number of small and hot files, that actually used to be
fully loaded in memory before we moved them off-heap. In the general case,
these files are expected to fully fit into the page cache for things to work
well. Should we give control over preloading to codecs? This is what this
commit does for the following files:
 - Terms index (`tip`)
 - Points index (`kdi`)
 - Stored fields index (`fdx`)
 - Terms vector index (`tvx`)

This only has an effect on `MMapDirectory`.
2022-11-15 13:59:51 +01:00
Robert Muir 1b9d98d6ec
enable error-prone "narrow calculation" check (#11923)
This check finds bugs such as https://github.com/apache/lucene/pull/11905.

See https://errorprone.info/bugpattern/NarrowCalculation
2022-11-15 06:42:41 -05:00
Lu Xugang e5426dbbd2
count() in BooleanQuery could be early quit (#11895)
* Count() in BooleanQuery could be early quit if queries are pure disjunctional
2022-11-15 17:57:51 +08:00
Adrien Grand 69a7cb22e7
More granular control of preloading on MMapDirectory. (#11929)
This enables configuring preloading on MMapDirectory based on the file name as well as the IOContext that is used to open the file.
2022-11-15 10:52:49 +01:00
Dawid Weiss d270025b7f
Update github actions to v3. (#11931) 2022-11-14 19:50:52 +01:00
Uwe Schindler 0741a354c0
Improve test introduced in #11918 to also check that reported invalid position is transformed back original position by slicing code (#11926) 2022-11-13 15:34:10 +01:00
Uwe Schindler 98b26e0885 fix merge problem in CHANGES.txt 2022-11-11 17:36:08 +01:00
Uwe Schindler 57ac311c70
Port generic exception handling from MemorySegmentIndexInput to ByteBufferIndexInput (#11918)
Port generic exception handling from MemorySegmentIndexInput to ByteBufferIndexInput. This also adds the invalid position while seeking or reading to the exception message.
2022-11-11 16:47:52 +01:00
Uwe Schindler 2a68f282f4 Synchronize changelog with 9.4 branch so we do not have duplicates 2022-11-11 16:36:02 +01:00
Peter Gromov 6fbc5f73c3
hunspell: introduce FragmentChecker to speed up ModifyingSuggester (#11909)
hunspell: introduce FragmentChecker to speed up ModifyingSuggester

add NGramFragmentChecker to quickly check whether insertions/replacements produce strings that are even possible in the language

Co-authored-by: Dawid Weiss <dawid.weiss@gmail.com>
2022-11-11 12:13:47 +01:00
Benjamin Trent c8d44acf20
Follow up to GITHUB#11916, remove deleted docs check (#11919) 2022-11-10 18:40:24 -05:00
Benjamin Trent 3a506ec87a
GITHUB#11911: improve checkindex to be more thorough for vectors (#11916)
search every N docs to get close to 64 tests
2022-11-10 16:45:47 -05:00
Uwe Schindler e9ef61ba39 Fix bug with set of strings since upgrade of Gradle -> explicit cast from GString to String 2022-11-10 17:18:30 +01:00
Benjamin Trent 1360baaee9
Fix integer overflow when seeking the vector index for connections (#11905)
* Fix integer overflow when seeking the vector index for connections
* Adding monster test to cause overflow failure
2022-11-10 08:24:32 -05:00
Peter Gromov f7417d5961
hunspell: allow for faster dictionary iteration during 'suggest' by using more memory (opt-in) (#11893)
hunspell: allow for faster dictionary iteration during 'suggest' by using more memory (opt-in)
2022-11-09 08:20:50 +01:00
Greg Miller c66a559050
Further optimize DrillSideways scoring (#11881) 2022-11-08 10:08:12 -08:00
Benjamin Trent f9c26ed501
Fix latent casting bug in BKDWriter (#11907) 2022-11-08 15:55:07 +01:00
Peter Gromov 682e5c94e8
[hunspell] speed up WordFormGenerator (#11904) 2022-11-07 19:41:17 +01:00
Lu Xugang a8120bcb32
Simplify the logic of matchAll() in IndexSortSortedNumericDocValuesRangeQuery (#11884)
* Simplify the logic of matchAll() in IndexSortSortedNumericDocValuesRangeQuery
2022-11-07 19:09:52 +08:00
Michael Sokolov 48aad5090f
#11896: reduce top k in test to avoid split-graph (#11899) 2022-11-04 09:30:46 -04:00
Nhat Nguyen 1a5ad61b9d
Document that bulkScorer method can return null (#11897)
Like Weight#scorer, we should warn users that Weight#bulkScorer can 
return null if the query matches no documents.
2022-11-02 15:12:43 -07:00
Robert Muir 4e207fed62
Tone down TestDocumentsWriterStallControl.testRandom, so it does not take minutes (#11894)
This test often takes several minutes with normal runs (no NIGHTLY/multiplier/etc). Tone it down so that it isn't slow: CI builds can work it harder by passing those parameters
2022-11-02 12:17:15 -04:00
Tim Stewart 7c130d2f07
Fix type in CONTRIBUTING.md (#11879) 2022-11-01 20:10:05 +00:00
Peter Gromov 419ffd3974 [hunspell] perform a bit fewer checks after 2 suffixes have been removed 2022-10-31 10:09:54 +01:00
Marios Trivyzas 3210a42f09
Fix nanos to millis conversion for tests (#11856) 2022-10-29 09:05:17 +02:00
Patrick Zhai 26ec0dd44c
add gradle aggregated coverage console log and html location (#11882)
* jacoco/ coverage shouldn't trigger all test tasks as dependencies - instead, it should run after those test tasks that you choose to run. removed java plugin from top-level.

* Make coverage depend on the default test task.

* Update jacoco log plugin so that it doesn't make hard dependencies on test tasks.

Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
2022-10-28 23:33:37 -07:00
Robert Muir 8736c18747
Allow building with java 18 now that gradle supports it (#11889)
* Allow building with java 18 now that gradle supports it
* update the "generic error" in these scripts
2022-10-28 23:41:09 -04:00
Navneet Verma e7253f112d
Add interface to relate a LatLonShape with another shape represented as Component2D. (#11753)
Adds createLatLonShapeDocValues and createXYShapeDocValues factory methods
to LatLonShape and XYShape factory classes, respectively.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-10-28 13:52:20 -05:00