From 79a2c253871e16ea8d426fd98806d3a6119934ae Mon Sep 17 00:00:00 2001 From: Jason Tedor Date: Fri, 5 Jan 2018 17:19:34 -0500 Subject: [PATCH] Update testing docs to reflect Gradle wrapper This commit updates the testing docs to reflect the usage of the Gradle wrapper. Relates #28107 --- GRADLE.CHEATSHEET | 4 +- README.textile | 4 +- TESTING.asciidoc | 120 +++++++++++++++++++++++----------------------- 3 files changed, 65 insertions(+), 63 deletions(-) diff --git a/GRADLE.CHEATSHEET b/GRADLE.CHEATSHEET index 2c9c34fe1b5..3e61c156d1d 100644 --- a/GRADLE.CHEATSHEET +++ b/GRADLE.CHEATSHEET @@ -1,4 +1,6 @@ -As a quick helper, below are the equivalent commands from maven to gradle (TESTING.md has also been updated). You can also run "gradle tasks" to see all tasks that are available to run. +As a quick helper, below are the equivalent commands from Maven to Gradle +(TESTING.md has also been updated). You can also run `./gradlew tasks` to see +all tasks that are available to run. clean -> clean test -> test verify -> check diff --git a/README.textile b/README.textile index 63c45abfe0e..f17958262d8 100644 --- a/README.textile +++ b/README.textile @@ -202,9 +202,9 @@ We have just covered a very small portion of what Elasticsearch is all about. Fo h3. Building from Source -Elasticsearch uses "Gradle":https://gradle.org for its build system. You'll need to have at least version 3.3 of Gradle installed. +Elasticsearch uses "Gradle":https://gradle.org for its build system. -In order to create a distribution, simply run the @gradle assemble@ command in the cloned directory. +In order to create a distribution, simply run the @./gradlew assemble@ command in the cloned directory. The distribution for each project will be created under the @build/distributions@ directory in that project. diff --git a/TESTING.asciidoc b/TESTING.asciidoc index 1cb05792b82..37f7962ff0c 100644 --- a/TESTING.asciidoc +++ b/TESTING.asciidoc @@ -13,7 +13,7 @@ To create a distribution without running the tests, simply run the following: ----------------------------- -gradle assemble +./gradlew assemble ----------------------------- === Running Elasticsearch from a checkout @@ -22,7 +22,7 @@ In order to run Elasticsearch from source without building a package, you can run it using Gradle: ------------------------------------- -gradle run +./gradlew run ------------------------------------- === Test case filtering. @@ -33,20 +33,20 @@ gradle run Run a single test case (variants) ---------------------------------------------------------- -gradle test -Dtests.class=org.elasticsearch.package.ClassName -gradle test "-Dtests.class=*.ClassName" +./gradlew test -Dtests.class=org.elasticsearch.package.ClassName +./gradlew test "-Dtests.class=*.ClassName" ---------------------------------------------------------- Run all tests in a package and sub-packages ---------------------------------------------------- -gradle test "-Dtests.class=org.elasticsearch.package.*" +./gradlew test "-Dtests.class=org.elasticsearch.package.*" ---------------------------------------------------- Run any test methods that contain 'esi' (like: ...r*esi*ze...). ------------------------------- -gradle test "-Dtests.method=*esi*" +./gradlew test "-Dtests.method=*esi*" ------------------------------- You can also filter tests by certain annotations ie: @@ -59,7 +59,7 @@ You can also filter tests by certain annotations ie: Those annotation names can be combined into a filter expression like: ------------------------------------------------ -gradle test -Dtests.filter="@nightly and not @backwards" +./gradlew test -Dtests.filter="@nightly and not @backwards" ------------------------------------------------ to run all nightly test but not the ones that are backwards tests. `tests.filter` supports @@ -67,7 +67,7 @@ the boolean operators `and, or, not` and grouping ie: --------------------------------------------------------------- -gradle test -Dtests.filter="@nightly and not(@badapple or @backwards)" +./gradlew test -Dtests.filter="@nightly and not(@badapple or @backwards)" --------------------------------------------------------------- === Seed and repetitions. @@ -75,7 +75,7 @@ gradle test -Dtests.filter="@nightly and not(@badapple or @backwards)" Run with a given seed (seed is a hex-encoded long). ------------------------------ -gradle test -Dtests.seed=DEADBEEF +./gradlew test -Dtests.seed=DEADBEEF ------------------------------ === Repeats _all_ tests of ClassName N times. @@ -84,7 +84,7 @@ Every test repetition will have a different method seed (derived from a single random master seed). -------------------------------------------------- -gradle test -Dtests.iters=N -Dtests.class=*.ClassName +./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName -------------------------------------------------- === Repeats _all_ tests of ClassName N times. @@ -93,7 +93,7 @@ Every test repetition will have exactly the same master (0xdead) and method-level (0xbeef) seed. ------------------------------------------------------------------------ -gradle test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.seed=DEAD:BEEF +./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.seed=DEAD:BEEF ------------------------------------------------------------------------ === Repeats a given test N times @@ -103,14 +103,14 @@ ie: testFoo[0], testFoo[1], etc... so using testmethod or tests.method ending in a glob is necessary to ensure iterations are run). ------------------------------------------------------------------------- -gradle test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.method=mytest* +./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.method=mytest* ------------------------------------------------------------------------- Repeats N times but skips any tests after the first failure or M initial failures. ------------------------------------------------------------- -gradle test -Dtests.iters=N -Dtests.failfast=true -Dtestcase=... -gradle test -Dtests.iters=N -Dtests.maxfailures=M -Dtestcase=... +./gradlew test -Dtests.iters=N -Dtests.failfast=true -Dtestcase=... +./gradlew test -Dtests.iters=N -Dtests.maxfailures=M -Dtestcase=... ------------------------------------------------------------- === Test groups. @@ -120,9 +120,9 @@ Test groups can be enabled or disabled (true/false). Default value provided below in [brackets]. ------------------------------------------------------------------ -gradle test -Dtests.nightly=[false] - nightly test group (@Nightly) -gradle test -Dtests.weekly=[false] - weekly tests (@Weekly) -gradle test -Dtests.awaitsfix=[false] - known issue (@AwaitsFix) +./gradlew test -Dtests.nightly=[false] - nightly test group (@Nightly) +./gradlew test -Dtests.weekly=[false] - weekly tests (@Weekly) +./gradlew test -Dtests.awaitsfix=[false] - known issue (@AwaitsFix) ------------------------------------------------------------------ === Load balancing and caches. @@ -132,7 +132,7 @@ want to explicitly specify the number of JVMs you can do so on the command line: ---------------------------- -gradle test -Dtests.jvms=8 +./gradlew test -Dtests.jvms=8 ---------------------------- Or in `~/.gradle/gradle.properties`: @@ -151,7 +151,7 @@ It is possible to provide a version that allows to adapt the tests behaviour to older features or bugs that have been changed or fixed in the meantime. ----------------------------------------- -gradle test -Dtests.compatibility=1.0.0 +./gradlew test -Dtests.compatibility=1.0.0 ----------------------------------------- @@ -160,50 +160,50 @@ gradle test -Dtests.compatibility=1.0.0 Run all tests without stopping on errors (inspect log files). ----------------------------------------- -gradle test -Dtests.haltonfailure=false +./gradlew test -Dtests.haltonfailure=false ----------------------------------------- Run more verbose output (slave JVM parameters, etc.). ---------------------- -gradle test -verbose +./gradlew test -verbose ---------------------- Change the default suite timeout to 5 seconds for all tests (note the exclamation mark). --------------------------------------- -gradle test -Dtests.timeoutSuite=5000! ... +./gradlew test -Dtests.timeoutSuite=5000! ... --------------------------------------- -Change the logging level of ES (not gradle) +Change the logging level of ES (not Gradle) -------------------------------- -gradle test -Dtests.es.logger.level=DEBUG +./gradlew test -Dtests.es.logger.level=DEBUG -------------------------------- Print all the logging output from the test runs to the commandline even if tests are passing. ------------------------------ -gradle test -Dtests.output=always +./gradlew test -Dtests.output=always ------------------------------ Configure the heap size. ------------------------------ -gradle test -Dtests.heap.size=512m +./gradlew test -Dtests.heap.size=512m ------------------------------ Pass arbitrary jvm arguments. ------------------------------ # specify heap dump path -gradle test -Dtests.jvm.argline="-XX:HeapDumpPath=/path/to/heapdumps" +./gradlew test -Dtests.jvm.argline="-XX:HeapDumpPath=/path/to/heapdumps" # enable gc logging -gradle test -Dtests.jvm.argline="-verbose:gc" +./gradlew test -Dtests.jvm.argline="-verbose:gc" # enable security debugging -gradle test -Dtests.jvm.argline="-Djava.security.debug=access,failure" +./gradlew test -Dtests.jvm.argline="-Djava.security.debug=access,failure" ------------------------------ == Backwards Compatibility Tests @@ -214,7 +214,7 @@ To run backwards compatibilty tests untar or unzip a release and run the tests with the following command: --------------------------------------------------------------------------- -gradle test -Dtests.filter="@backwards" -Dtests.bwc.version=x.y.z -Dtests.bwc.path=/path/to/elasticsearch -Dtests.security.manager=false +./gradlew test -Dtests.filter="@backwards" -Dtests.bwc.version=x.y.z -Dtests.bwc.path=/path/to/elasticsearch -Dtests.security.manager=false --------------------------------------------------------------------------- Note that backwards tests must be run with security manager disabled. @@ -222,7 +222,7 @@ If the elasticsearch release is placed under `./backwards/elasticsearch-x.y.z` t can be omitted: --------------------------------------------------------------------------- -gradle test -Dtests.filter="@backwards" -Dtests.bwc.version=x.y.z -Dtests.security.manager=false +./gradlew test -Dtests.filter="@backwards" -Dtests.bwc.version=x.y.z -Dtests.security.manager=false --------------------------------------------------------------------------- To setup the bwc test environment execute the following steps (provided you are @@ -239,20 +239,20 @@ $ tar -xzf elasticsearch-1.2.1.tar.gz To run all verification tasks, including static checks, unit tests, and integration tests: --------------------------------------------------------------------------- -gradle check +./gradlew check --------------------------------------------------------------------------- Note that this will also run the unit tests and precommit tasks first. If you want to just run the integration tests (because you are debugging them): --------------------------------------------------------------------------- -gradle integTest +./gradlew integTest --------------------------------------------------------------------------- If you want to just run the precommit checks: --------------------------------------------------------------------------- -gradle precommit +./gradlew precommit --------------------------------------------------------------------------- == Testing the REST layer @@ -264,18 +264,18 @@ The REST layer is tested through specific tests that are shared between all the elasticsearch official clients and consist of YAML files that describe the operations to be executed and the obtained results that need to be tested. -The REST tests are run automatically when executing the "gradle check" command. To run only the +The REST tests are run automatically when executing the "./gradlew check" command. To run only the REST tests use the following command: --------------------------------------------------------------------------- -gradle :distribution:integ-test-zip:integTest \ +./gradlew :distribution:integ-test-zip:integTest \ -Dtests.class="org.elasticsearch.test.rest.*Yaml*IT" --------------------------------------------------------------------------- A specific test case can be run with --------------------------------------------------------------------------- -gradle :distribution:integ-test-zip:integTest \ +./gradlew :distribution:integ-test-zip:integTest \ -Dtests.class="org.elasticsearch.test.rest.*Yaml*IT" \ -Dtests.method="test {p0=cat.shards/10_basic/Help}" --------------------------------------------------------------------------- @@ -319,25 +319,25 @@ vagrant plugin install vagrant-cachier . Validate your installed dependencies: ------------------------------------- -gradle :qa:vagrant:vagrantCheckVersion +./gradlew :qa:vagrant:vagrantCheckVersion ------------------------------------- -. Download and smoke test the VMs with `gradle vagrantSmokeTest` or -`gradle -Pvagrant.boxes=all vagrantSmokeTest`. The first time you run this it will +. Download and smoke test the VMs with `./gradlew vagrantSmokeTest` or +`./gradlew -Pvagrant.boxes=all vagrantSmokeTest`. The first time you run this it will download the base images and provision the boxes and immediately quit. If you you this again it'll skip the download step. -. Run the tests with `gradle packagingTest`. This will cause gradle to build +. Run the tests with `./gradlew packagingTest`. This will cause Gradle to build the tar, zip, and deb packages and all the plugins. It will then run the tests on ubuntu-1404 and centos-7. We chose those two distributions as the default because they cover deb and rpm packaging and SyvVinit and systemd. -You can run on all the VMs by running `gradle -Pvagrant.boxes=all packagingTest`. -You can run a particular VM with a command like -`gradle -Pvagrant.boxes=oel-7 packagingTest`. See `gradle tasks` for a complete -list of available vagrant boxes for testing. It's important to know that if you -ctrl-c any of these `gradle` commands then the boxes will remain running and -you'll have to terminate them with 'gradle stop'. +You can run on all the VMs by running `./gradlew -Pvagrant.boxes=all +packagingTest`. You can run a particular VM with a command like `./gradlew +-Pvagrant.boxes=oel-7 packagingTest`. See `./gradlew tasks` for a complete list +of available vagrant boxes for testing. It's important to know that if you +interrupt any of these Gradle commands then the boxes will remain running and +you'll have to terminate them with `./gradlew stop`. All the regular vagrant commands should just work so you can get a shell in a VM running trusty by running @@ -402,10 +402,10 @@ that'd consume a ton of ram. In general its best to stick to testing in vagrant because the bats scripts are destructive. When working with a single package it's generally faster to run its -tests in a tighter loop than gradle provides. In one window: +tests in a tighter loop than Gradle provides. In one window: -------------------------------- -gradle :distribution:rpm:assemble +./gradlew :distribution:rpm:assemble -------------------------------- and in another window: @@ -419,7 +419,7 @@ sudo -E bats $BATS_TESTS/*rpm*.bats If you wanted to retest all the release artifacts on a single VM you could: ------------------------------------------------- -gradle setupBats +./gradlew setupBats cd qa/vagrant; vagrant up ubuntu-1404 --provider virtualbox && vagrant ssh ubuntu-1404 cd $BATS_ARCHIVES sudo -E bats $BATS_TESTS/*.bats @@ -428,7 +428,7 @@ sudo -E bats $BATS_TESTS/*.bats You can also use Gradle to prepare the test environment and then starts a single VM: ------------------------------------------------- -gradle vagrantFedora27#up +./gradlew vagrantFedora27#up ------------------------------------------------- Or any of vagrantCentos6#up, vagrantCentos7#up, vagrantDebian8#up, @@ -457,17 +457,17 @@ Backwards compatibility tests exist to test upgrading from each supported versio to the current version. To run all backcompat tests use: ------------------------------------------------- -gradle bwcTest +./gradlew bwcTest ------------------------------------------------- A specific version can be tested as well. For example, to test backcompat with version 5.3.2 run: ------------------------------------------------- -gradle v5.3.2#bwcTest +./gradlew v5.3.2#bwcTest ------------------------------------------------- -When running `gradle check`, some minimal backcompat checks are run. Which version +When running `./gradlew check`, some minimal backcompat checks are run. Which version is tested depends on the branch. On master, this will test against the current stable branch. On the stable branch, it will test against the latest release branch. Finally, on a release branch, it will test against the most recent release. @@ -476,11 +476,11 @@ branch. Finally, on a release branch, it will test against the most recent relea Sometimes a backward compatibility change spans two versions. A common case is a new functionality that needs a BWC bridge in and an unreleased versioned of a release branch (for example, 5.x). -To test the changes, you can instruct gradle to build the BWC version from a another remote/branch combination instead of +To test the changes, you can instruct Gradle to build the BWC version from a another remote/branch combination instead of pulling the release branch from GitHub. You do so using the `tests.bwc.remote` and `tests.bwc.refspec` system properties: ------------------------------------------------- -gradle check -Dtests.bwc.remote=${remote} -Dtests.bwc.refspec=index_req_bwc_5.x +./gradlew check -Dtests.bwc.remote=${remote} -Dtests.bwc.refspec=index_req_bwc_5.x ------------------------------------------------- The branch needs to be available on the remote that the BWC makes of the @@ -495,7 +495,7 @@ will need to: will contain your change. . Create a branch called `index_req_bwc_5.x` off `5.x`. This will contain your bwc layer. . Push both branches to your remote repository. -. Run the tests with `gradle check -Dtests.bwc.remote=${remote} -Dtests.bwc.refspec=index_req_bwc_5.x`. +. Run the tests with `./gradlew check -Dtests.bwc.remote=${remote} -Dtests.bwc.refspec=index_req_bwc_5.x`. == Coverage analysis @@ -522,11 +522,11 @@ mvn -Dtests.coverage verify jacoco:report == Launching and debugging from an IDE -If you want to run elasticsearch from your IDE, the `gradle run` task +If you want to run elasticsearch from your IDE, the `./gradlew run` task supports a remote debugging option: --------------------------------------------------------------------------- -gradle run --debug-jvm +./gradlew run --debug-jvm --------------------------------------------------------------------------- == Debugging remotely from an IDE @@ -549,7 +549,7 @@ and the build will automatically pick it up. You can verify the plugin is included as part of the build by checking the projects of the build. --------------------------------------------------------------------------- -gradle projects +./gradlew projects --------------------------------------------------------------------------- == Environment misc