From a843058f5f458d9950ad9468afcc22b94fd43ddb Mon Sep 17 00:00:00 2001 From: Michael Stack Date: Thu, 19 Apr 2012 16:24:28 +0000 Subject: [PATCH] Some more info on test categorization; add the rule that needs to at foot of all new test files git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1328035 13f79535-47bb-0310-9956-ffa450edef68 --- src/docbkx/developer.xml | 51 +++++++++++++++++++++++++++++----------- 1 file changed, 37 insertions(+), 14 deletions(-) diff --git a/src/docbkx/developer.xml b/src/docbkx/developer.xml index dd63e2a3a00..bffa0dc2ff9 100644 --- a/src/docbkx/developer.xml +++ b/src/docbkx/developer.xml @@ -240,9 +240,10 @@ What is the new development version for "HBase"? (org.apache.hbase:hbase) 0.92.1 Tests HBase tests are divided into two groups: and -. -Unit tests are run by the Apache Continuous Integration server and by developers -when they are verifying a fix does not cause breakage elsewhere in the code base. + (As of this writing, Integration tests are little +developed). +Unit tests are run by the Apache Continuous Integration server, Jenkins at +builds.apache.org, and by developers when they are verifying a fix does not cause breakage elsewhere in the code base. Integration tests are generally long-running tests that are invoked out-of-bound of the CI server when you want to do more intensive testing beyond the unit test set. Integration tests, for example, are run proving a release candidate or a production @@ -265,24 +266,42 @@ public class TestHRegionInfo { public void testCreateHRegionInfoName() throws Exception { // ... } +... + @org.junit.Rule + public org.apache.hadoop.hbase.ResourceCheckerJUnitRule cu = + new org.apache.hadoop.hbase.ResourceCheckerJUnitRule(); } -The above example shows how to mark a test as belonging to the small category. +The above example shows how to mark a test as belonging to the small category. The @org.junit.Rule +lines on the end are also necessary. Add them to each new unit test file. They are needed by the categorization process. +HBase uses a patched maven surefire plugin and maven profiles to implement its unit test characterizations. + +
+<indexterm><primary>SmallTests</primary></indexterm> Small tests are executed in a shared JVM. We put in this category all the tests that can -be executed quickly in a shared JVM. The maximum execution time for a test is 15 seconds, -and they do not use a cluster. Medium tests represent tests that must be executed +be executed quickly in a shared JVM. The maximum execution time for a small test is 15 seconds, +and small tests should not use a (mini)cluster. +
+ +
+<indexterm><primary>MediumTests</primary></indexterm> +Medium tests represent tests that must be executed before proposing a patch. They are designed to run in less than 30 minutes altogether, and are quite stable in their results. They are designed to last less than 50 seconds individually. They can use a cluster, and each of them is executed in a separate JVM. -Large tests are everything else. They are typically integration-like + +
+ +
+<indexterm><primary>LargeTests</primary></indexterm> +Large tests are everything else. They are typically integration-like tests (yes, some large tests should be moved out to be HBase ), regression tests for specific bugs, timeout tests, performance tests. They are executed before a commit on the pre-integration machines. They can be run on the developer machine as well. -HBase uses a patched maven surefire plugin and maven profiles to implement its -unit test characterizations. +
Running tests @@ -291,8 +310,9 @@ unit test characterizations.
Default: small and medium category tests -Running mvn test will execute all small tests in a single JVM and medium tests in a separate JVM for -each test instance. Medium tests are NOT executed if there is an error in a small test. +Running mvn test will execute all small tests in a single JVM +(no fork) and then medium tests in a separate JVM for each test instance. +Medium tests are NOT executed if there is an error in a small test. Large tests are NOT executed. There is one report for small tests, and one report for medium tests if they are executed. To run small and medium tests with the security profile enabled, do mvn test -P security @@ -323,9 +343,10 @@ To run a single test with the security profile enabled: The -P localTests will remove the JUnit category effect (without this specific profile, -the profiles are taken into account). It will actually use the official release of surefire +the categories are taken into account). It will actually use the official release of surefire and the old connector (The HBase build uses a patched version of the maven surefire plugin). -junit tests are executed in separated JVM. You will see a new message at the end of the +Each junit tests is executed in a separate JVM (A fork per test class). There is no +parallelization when localTests profile is set. You will see a new message at the end of the report: "[INFO] Tests are skipped". It's harmless.
@@ -343,7 +364,9 @@ report: "[INFO] Tests are skipped". It's harmless.
<command>hbasetests.sh</command> It's also possible to use the script hbasetests.sh. This script runs the medium and -large tests in parallel with two maven instances, and provide a single report. +large tests in parallel with two maven instances, and provides a single report. This script does not use +the hbase version of surefire so no parallelization is being done other than the two maven instances the +script sets up. It must be executed from the directory which contains the pom.xml. For example running ./dev-support/hbasetests.sh will execute small and medium tests.