lucene/help/tests.txt

Testing
=======

Examples below assume cwd at the gradlew script in the top directory of
the project's checkout.


Generic test/ checkup commands
------------------------------

Run all unit tests:

gradlew test

Run all verification tasks, including tests:

gradlew check

Run all verification tasks, excluding tests (-x is gradle's generic task
exclusion mechanism):

gradlew check -x test

Run verification for a selected module only:

gradlew :lucene:core:check     # By full gradle project path
gradlew -p lucene/core check   # By folder designation + task


Randomization
-------------

To run tests with the given starting seed pass 'tests.seed'
property:

gradlew :lucene:misc:test -Ptests.seed=DEADBEEF

There are a lot of other test randomization properties
available. To list them, their defaults and current values
run the testOpts task against a project that has tests.
For example:

gradlew -p lucene/core testOpts


Filtering
---------

Run tests of lucene-core module:

gradlew -p lucene/core test

Run a single test case (from a single module). Uses gradle's built-in filtering
(https://docs.gradle.org/current/userguide/java_testing.html#test_filtering):

gradlew -p lucene/core test --tests TestDemo

Run all tests in a package:

gradlew -p lucene/core test --tests "org.apache.lucene.document.*"

Run all test classes/ methods that match this pattern:

gradlew -p lucene/core test --tests "*testFeatureMissing*"


Test groups
-----------

Tests can be filtered by an annotation they're marked with.
Some test group annotations include: @AwaitsFix, @Nightly, @Slow

This uses filtering infrastructure on the *runner* (randomizedtesting), 
not gradle's built-in mechanisms (but it can be combined with "--tests").
For example, run all lucene-core tests annotated as @Slow:

gradlew -p lucene/core test -Ptests.filter=@Slow

Test group filters can be combined into Boolean expressions:

gradlew -p lucene/core test "default and not(@awaitsfix or @slow)"


Reiteration ("beasting")
------------------------

Multiply each test case N times (this works by repeating the same test
within the same JVM; it also works in IDEs):

gradlew -p lucene/core test --tests TestDemo -Ptests.iters=5

Tests tasks will be (by default) re-executed on each invocation because
we pick a random global tests.seed. If you run the same tests twice
with the same seed, the test task will be skipped (as it is up-to-date
with respect to source code):

gradlew -p lucene/core test -Ptests.seed=deadbeef

to force re-execution of tests, even for the same master seed, apply
cleanTest task:

gradlew -p lucene/core cleanTest test -Ptests.seed=deadbeef


Verbose mode and debugging
--------------------------

The "tests.verbose" mode switch enables standard streams from tests
to be dumped directly to the console. Run your verbose tests explicitly
specifying the project and test task or a fully qualified task path. Example:

gradlew -p lucene/core test -Ptests.verbose=true --tests "TestDemo"


Testing against different JVMs
------------------------------

By default tests are executed with the same Java gradle is using internally.
To run tests against a different Java version define a property called
"runtime.java.home" or define an environment variable "RUNTIME_JAVA_HOME"
pointing at the JDK installation folder.

If property is used, it can be a system property (-D...) or a project
property (-P...).

Example:

gradlew test -p lucene/test-framework --tests TestJvmInfo -Dtests.verbose=true -Druntime.java.home=/jvms/jdk14