mirror of https://github.com/apache/lucene.git
158 lines
4.5 KiB
Plaintext
158 lines
4.5 KiB
Plaintext
Testing
|
|
=======
|
|
|
|
Examples below assume cwd at the gradlew script in the top directory of
|
|
the project's checkout.
|
|
|
|
|
|
Generic test/ checkup commands
|
|
------------------------------
|
|
|
|
Run all unit tests:
|
|
|
|
gradlew test
|
|
|
|
Run all verification tasks, including tests:
|
|
|
|
gradlew check
|
|
|
|
Run all verification tasks, excluding tests (-x is gradle's generic task
|
|
exclusion mechanism):
|
|
|
|
gradlew check -x test
|
|
|
|
Run verification for a selected module only:
|
|
|
|
gradlew :lucene:core:check # By full gradle project path
|
|
gradlew -p lucene/core check # By folder designation + task
|
|
|
|
|
|
Randomization
|
|
-------------
|
|
|
|
To run tests with the given starting seed pass 'tests.seed'
|
|
property:
|
|
|
|
gradlew :lucene:misc:test -Ptests.seed=DEADBEEF
|
|
|
|
There are a lot of other test randomization properties
|
|
available. To list them, their defaults and current values
|
|
run the testOpts task against a project that has tests.
|
|
For example:
|
|
|
|
gradlew -p lucene/core testOpts
|
|
|
|
|
|
Filtering
|
|
---------
|
|
|
|
Run tests of lucene-core module:
|
|
|
|
gradlew -p lucene/core test
|
|
|
|
Run a single test case (from a single module). Uses gradle's built-in filtering
|
|
(https://docs.gradle.org/current/userguide/java_testing.html#test_filtering):
|
|
|
|
gradlew -p lucene/core test --tests TestDemo
|
|
|
|
Run all tests in a package:
|
|
|
|
gradlew -p lucene/core test --tests "org.apache.lucene.document.*"
|
|
|
|
Run all test classes/ methods that match this pattern:
|
|
|
|
gradlew -p lucene/core test --tests "*testFeatureMissing*"
|
|
|
|
|
|
Test groups
|
|
-----------
|
|
|
|
Tests can be filtered by an annotation they're marked with.
|
|
Some test group annotations include: @AwaitsFix, @Nightly, @Slow
|
|
|
|
This uses filtering infrastructure on the *runner* (randomizedtesting),
|
|
not gradle's built-in mechanisms (but it can be combined with "--tests").
|
|
For example, run all lucene-core tests annotated as @Slow:
|
|
|
|
gradlew -p lucene/core test -Ptests.filter=@Slow
|
|
|
|
Test group filters can be combined into Boolean expressions:
|
|
|
|
gradlew -p lucene/core test "default and not(@awaitsfix or @slow)"
|
|
|
|
|
|
Reiteration ("beasting")
|
|
------------------------
|
|
|
|
Multiply each test case N times (this works by repeating the same test
|
|
within the same JVM; it also works in IDEs):
|
|
|
|
gradlew -p lucene/core test --tests TestDemo -Ptests.iters=5
|
|
|
|
Tests tasks will be (by default) re-executed on each invocation because
|
|
we pick a random global tests.seed. If you run the same tests twice
|
|
with the same seed, the test task will be skipped (as it is up-to-date
|
|
with respect to source code):
|
|
|
|
gradlew -p lucene/core test -Ptests.seed=deadbeef
|
|
|
|
to force re-execution of tests, even for the same master seed, apply
|
|
cleanTest task:
|
|
|
|
gradlew -p lucene/core cleanTest test -Ptests.seed=deadbeef
|
|
|
|
|
|
Verbose mode and debugging
|
|
--------------------------
|
|
|
|
The "tests.verbose" mode switch enables standard streams from tests
|
|
to be dumped directly to the console. Run your verbose tests explicitly
|
|
specifying the project and test task or a fully qualified task path. Example:
|
|
|
|
gradlew -p lucene/core test -Ptests.verbose=true --tests "TestDemo"
|
|
|
|
|
|
Profiling slow tests
|
|
--------------------
|
|
|
|
The "tests.profile" mode switch turns on a sampling profiler during test execution,
|
|
and prints a simple summary at the end.
|
|
|
|
For example, top-10 histogram of methods (cpu samples):
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true
|
|
|
|
Alternatively, you can profile heap allocations instead:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.mode=heap
|
|
|
|
By default, results are computed (deduplicated) on just the method name, folding
|
|
together all events from the same method. To drill down further, you can increase the
|
|
stack size from the default of 1, to get a histogram of stacktraces instead:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.stacksize=8
|
|
|
|
For big methods, it can also be helpful to include line numbers for more granularity:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.linenumbers=true
|
|
|
|
Using these additional options will make the results more sparse, so it may be useful
|
|
to increase the top-N count:
|
|
|
|
gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.count=100
|
|
|
|
Testing against different JVMs
|
|
------------------------------
|
|
|
|
By default tests are executed with the same Java gradle is using internally.
|
|
To run tests against a different Java version define a property called
|
|
"runtime.java.home" or define an environment variable "RUNTIME_JAVA_HOME"
|
|
pointing at the JDK installation folder.
|
|
|
|
If property is used, it can be a system property (-D...) or a project
|
|
property (-P...).
|
|
|
|
Example:
|
|
|
|
gradlew test -p lucene/test-framework --tests TestJvmInfo -Dtests.verbose=true -Druntime.java.home=/jvms/jdk14
|