Commit Graph

21 Commits

Author SHA1 Message Date
Michael McCandless afb2a60751
Bound the RAM used by the NodeHash (sharing common suffixes) during FST compilation (#12633)
* tweak comments; change if to switch

* remove old SOPs, minor comment styling, fixed silly performance bug on rehash using the wrong bitsRequired (count vs node)

* first raw cut; some nocommits added; some tests fail

* tests pass!

* fix silly fallback hash bug

* remove SOPs; add some temporary debugging metrics

* add temporary tool to test FST performance across differing NodeHash sizes

* remove (now deleted) shouldShareNonSingletonNodes call from Lucene90BlockTreeTermsWriter

* add simple tool to render results table to GitHub MD

* add simple temporary tool to iterate all terms from a provided luceneutil wikipedia index and build an FST from them

* first cut at using packed ints for hash t able again

* add some nocommits; tweak test_all_sizes.py to new RAM usage approach; when half of the double barrel is full, allocate new primary hash at full size to save cost of continuously rehashing for a large FST

* switch to limit suffix hash by RAM usage not count (more intuitive for users); clean up some stale nocommits

* switch to more intuitive approximate RAM (mb) limit for allowed size of NodeHash

* nuke a few nocommits; a few more remain

* remove DO_PRINT_HASH_RAM

* no more FST pruning

* remove final nocommit: randomly change allowed NodeHash suffix RAM size in TestFSTs.testRealTerms

* remove SOP

* tidy

* delete temp utility tools

* remove dead (FST pruning) code

* add CHANGES entry; fix one missed fst.addNode -> fstCompiler.addNode during merge conflict resolution

* remove a mal-formed nocommit

* fold PR feedback

* fold feedback

* add gradle help test details on how to specify heap size for the test JVM; fix bogus assert (uncovered by Test2BFST); add TODO to Test2BFST anticipating building massive FSTs in small bounded RAM

* suppress sysout checks for Test2BFSTs; add helpful comment showing how to run it directly

* tidy
2023-10-20 11:52:55 -04:00
Dawid Weiss 326142c485
Add a note about gradle checks being possibly a subset of all validation checks. (#11966) 2022-11-23 12:19:28 +01:00
Robert Muir 3edfeb5eb2
LUCENE-10532: remove @Slow annotation (#832)
Remove `@Slow` annotation, for more consistency with CI and local jobs. All tests can be fast!
2022-05-09 23:03:55 -04:00
Robert Muir 1089b482fc
LUCENE-10528: use Xvfb in test to avoid messing up user's desktop (#828)
Co-authored-by: Tomoko Uchida <tomoko.uchida.1111@gmail.com>
2022-04-23 08:00:33 -04:00
Dawid Weiss 486141f0eb LUCENE-9660: correct help/tests.txt. 2021-10-26 08:45:58 +02:00
Nhat Nguyen 9a17d67658
LUCENE-9935: Enable bulk merge for stored fields with index sort (#134)
This commit enables bulk-merges (i.e., raw chunk copying) for stored 
fields when index sort is enabled
2021-05-12 21:00:18 -04:00
Robert Muir 06907a2c12
LUCENE-9188: Add jacoco code coverage support to gradle (#119)
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
Co-authored-by: Uwe Schindler <uschindler@apache.org>
2021-05-02 16:24:06 +02:00
Dawid Weiss 246c4beb22
LUCENE-9854: Clean up utilities to download and extract test/ benchmark data sets. (#27) 2021-03-22 12:22:39 +01:00
Dawid Weiss 83ed210fd0
LUCENE-9465: 'beast' task from within gradle (#1757) 2020-08-18 09:28:50 +02:00
Dawid Weiss 8ebf2d0b21 LUCENE-9312: Allow builds against arbitrary JVMs (squashed
jira/LUCENE-9312)
2020-07-21 09:19:38 +02:00
Robert Muir ed7f507c3c
LUCENE-9193: fix documentation typo for gradle tests 2020-01-30 23:54:31 -05:00
Robert Muir 4b5105e167
LUCENE-9193: heap allocations for tests.profile
Can be a bit noisier than cpu sampling, due to how threads are allocated
in tests... maybe we can improve that in the future.
2020-01-30 08:29:10 -05:00
Robert Muir e504798a44
LUCENE-9185: add "tests.profile" to gradle build to aid fixing slow tests
Run test(s) with -Ptests.profile=true to print a histogram at the end of
the build.
2020-01-28 11:27:18 -05:00
Robert Muir 8e357b167b
LUCENE-9180: dos2unix files that don't need dos line endings 2020-01-27 11:29:59 -05:00
Cassandra Targett 74e88deba7 Revert "SOLR-12930: move Gradle docs from ./help/ to new ./dev-docs/ directory"
This reverts commit 2d8650d36c.
2020-01-24 15:56:00 -06:00
Cassandra Targett 2d8650d36c SOLR-12930: move Gradle docs from ./help/ to new ./dev-docs/ directory 2020-01-23 14:04:19 -06:00
Dawid Weiss 4599c51f0d LUCENE-9122: add support for running tests against alternate jvms. 2020-01-09 19:00:32 +01:00
Dawid Weiss 02c79dd211 Add testOpts task and info about it in tests.txt 2019-12-08 18:45:41 +01:00
Dawid Weiss 1021f04d1a Add some support for -Ptests.verbose mode when streams are dumped to the console. This is constrained by gradle's runner but is better than nothing. 2019-12-07 14:53:13 +01:00
Dawid Weiss 5459a7938f Clarify when test target skips and how to force-run it even for the same seed. 2019-12-02 18:04:56 +01:00
Dawid Weiss d4a9842375 Initial gradle build layer. 2019-12-02 15:34:57 +01:00