mirror of https://github.com/apache/lucene.git
158 lines
4.9 KiB
Plaintext
158 lines
4.9 KiB
Plaintext
Testing
|
|
=======
|
|
|
|
Examples below assume cwd at the gradlew script in the top directory of
|
|
the project's checkout.
|
|
|
|
|
|
Generic test/ checkup commands
|
|
------------------------------
|
|
|
|
Run all unit tests:
|
|
|
|
gradlew test
|
|
|
|
Run all verification tasks, including tests:
|
|
|
|
gradlew check
|
|
|
|
Run all verification tasks, excluding tests (-x is gradle's generic task
|
|
exclusion mechanism):
|
|
|
|
gradlew check -x test
|
|
|
|
Run verification for a selected module only:
|
|
|
|
gradlew :lucene:core:check # By full gradle project path
|
|
gradlew -p lucene/core check # By folder designation + task
|
|
|
|
|
|
Randomization
|
|
-------------
|
|
|
|
To run tests with the given starting seed pass 'tests.seed'
|
|
property:
|
|
|
|
gradlew :lucene:misc:test -Ptests.seed=DEADBEEF
|
|
|
|
There are a lot of other test randomization properties
|
|
available. To list them, their defaults and current values
|
|
run the testOpts task against a project that has tests.
|
|
For example:
|
|
|
|
gradlew -p lucene/core testOpts
|
|
|
|
|
|
Filtering
|
|
---------
|
|
|
|
Run tests of lucene-core module:
|
|
|
|
gradlew -p lucene/core test
|
|
|
|
Run a single test case (from a single module). Uses gradle's built-in filtering
|
|
(https://docs.gradle.org/current/userguide/java_testing.html#test_filtering):
|
|
|
|
gradlew -p lucene/core test --tests TestDemo
|
|
|
|
Run all tests in a package:
|
|
|
|
gradlew -p lucene/core test --tests "org.apache.lucene.document.*"
|
|
|
|
Run all test classes/ methods that match this pattern:
|
|
|
|
gradlew -p lucene/core test --tests "*testFeatureMissing*"
|
|
|
|
|
|
Test groups
|
|
-----------
|
|
|
|
Tests can be filtered by an annotation they're marked with.
|
|
Some test group annotations include: @AwaitsFix, @Nightly, @Slow
|
|
|
|
This uses filtering infrastructure on the *runner* (randomizedtesting),
|
|
not gradle's built-in mechanisms (but it can be combined with "--tests").
|
|
For example, run all lucene-core tests annotated as @Slow:
|
|
|
|
gradlew -p lucene/core test -Ptests.filter=@Slow
|
|
|
|
Test group filters can be combined into Boolean expressions:
|
|
|
|
gradlew -p lucene/core test "default and not(@awaitsfix or @slow)"
|
|
|
|
|
|
Reiteration ("beasting")
|
|
------------------------
|
|
|
|
Multiply each test case N times (this works by repeating the same test
|
|
within the same JVM; it also works in IDEs):
|
|
|
|
gradlew -p lucene/core test --tests TestDemo -Ptests.iters=5
|
|
|
|
Tests tasks will be (by default) re-executed on each invocation because
|
|
we pick a random global tests.seed. If you run the same tests twice
|
|
with the same seed, the test task will be skipped (as it is up-to-date
|
|
with respect to source code):
|
|
|
|
gradlew -p lucene/core test -Ptests.seed=deadbeef
|
|
|
|
to force re-execution of tests, even for the same master seed, apply
|
|
cleanTest task:
|
|
|
|
gradlew -p lucene/core cleanTest test -Ptests.seed=deadbeef
|
|
|
|
The 'tests.iters' option should be sufficient for individual test cases
|
|
and is *much* faster than trying to duplicate re-runs of the entire
|
|
test suites. When it is absolutely needed to re-run an entire suite (because
|
|
of randomization in the static initialization, for example), you can do it
|
|
by running the 'beast' task with 'tests.dups' option:
|
|
|
|
gradlew -p lucene/core beast -Ptests.dups=10 --tests TestPerFieldDocValuesFormat
|
|
|
|
Note the filter (--tests) used to narrow down test reiterations to a particular
|
|
class. You can use any filter, including no filter at all, but it rarely makes
|
|
sense (will take ages). By default the test tasks generated by the 'beast' mode
|
|
use a random starting seed for randomization. If you pass an explicit seed, this
|
|
won't be the case (all tasks will use exactly the same starting seed):
|
|
|
|
gradlew -p lucene/core beast -Ptests.dups=10 --tests TestPerFieldDocValuesFormat -Dtests.seed=deadbeef
|
|
|
|
Verbose mode and debugging
|
|
--------------------------
|
|
|
|
The "tests.verbose" mode switch enables standard streams from tests
|
|
to be dumped directly to the console. Run your verbose tests explicitly
|
|
specifying the project and test task or a fully qualified task path. Example:
|
|
|
|
gradlew -p lucene/core test -Ptests.verbose=true --tests "TestDemo"
|
|
|
|
|
|
Profiling slow tests
|
|
--------------------
|
|
|
|
The "tests.profile" mode switch turns on a sampling profiler during test execution,
|
|
and prints a simple summary at the end.
|
|
|
|
For example, top-10 histogram of methods (cpu samples):
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true
|
|
|
|
Alternatively, you can profile heap allocations instead:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.mode=heap
|
|
|
|
By default, results are computed (deduplicated) on just the method name, folding
|
|
together all events from the same method. To drill down further, you can increase the
|
|
stack size from the default of 1, to get a histogram of stacktraces instead:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.stacksize=8
|
|
|
|
For big methods, it can also be helpful to include line numbers for more granularity:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.linenumbers=true
|
|
|
|
Using these additional options will make the results more sparse, so it may be useful
|
|
to increase the top-N count:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.count=100
|