# Maven Structure The integration tests are built and run as part of Druid's Maven script. Maven itself is used by hand, and as part of the [Travis](travis.md) build proces. Running integration tests in maven is a multi-part process. * Build the product `distribution`. * Build the test image. The tests run against the Maven-created Druid build, and so appear in the root `pom.xml` file *after* the `distribution` project which builds the Druid tarball. * Run one or more ITs. Each Maven run includes a single test category and its required Druid cluster. Travis orchestrates the above process to run the ITs in parallel. When you run tests locally, you do the above steps one by one. You can, of course, reuse the same disribution for multiple image builds, and the same image for multiple test runs. ## Build the Distribution and Image Use the following command to run the ITs, assuming `DRUID_DEV` points to your Druid development directory: ```bash cd $DRUID_DEV mvn clean package -P dist,test-image,skip-static-checks \ -Dmaven.javadoc.skip=true -DskipUTs=true ``` The various pieces are: * `clean`: Remove any existing artifacts, and any existing Docker image. * `install`: Build the Druid code and write it to the local Maven repo. * `-P dist`: Create the Druid distribution tarball by pulling jars from the local Maven repo. * `-P test-image`: Build the Docker images by grabbing the Druid tarball and pulling additional dependencies into the local repo, then stage them for Docker. * Everything else: ignore parts of the build not needed for the ITs, such as static checks, unit tests, Javadoc, etc. Once you've done the above once, you can do just the specific part you want to repeat during development. See below for details. See [quickstart](quickstart.md) for how to run the two steps separately. ## Run Each Integration Test Category Each pass through Maven runs a single test category. Running a test category has three parts, spelled out in Maven: * Launch the required cluster. * Run the test category. * Shut down the cluster. Again, see [quickstart](quickstart.md) for how to run the three steps separately, and how to run the tests in an IDE. To do the task via Maven: ```bash cd $DRUID_DEV mvn verify -P docker-tests,skip-static-checks,IT- \ -Dmaven.javadoc.skip=true -DskipUTs=true ``` The various pieces are: * `verify`: Run the steps up to the one that checks the output of the ITs. Because of the extra cluster step in an IT, the build does not fail if an IT failse. Instead, it continues on to clean up the cluster, and only after that does it check test sucess in the `verify` step. * `` selects the category to run. The profiles allow you to build the test image once during debugging, and reuse it across multiple test runs. (See [Debugging](debugging.md).) ## Dependencies The Docker image inclues three third-party dependencies not included in the Druid build: * MySQL connector * MariaDB connector * Kafka Protobuf provider We use dependency rules in the `test-image/pom.xml` file to cause Maven to download these dependencies into the Maven cache, then we use the `maven-dependency-plugin` to copy those dependencies into a Docker directory, and we use Docker to copy the files into the image. This approach avoids the need to pull the dependency from a remote repository into the image directly, and thus both speeds up the build, and is kinder to the upstream repositories. If you add additional dependencies, please follow the above process. See the `pom.xml` files for examples. ## Environment Variables The build environment users environment variables to pass information to Maven. Maven communicates with Docker and Docker Compose via environment variables set in the `exec-maven-plugin` of various `pom.xml` files. The environment variables then flow into either the Docker build script (`Dockerfile`) or the various Docker Compose scripts (`docker-compose.yaml`). It can be tedious to follow this flow. A quick outline: * The build environment (such as Travis) sets environment variables, or passes values to maven via the `-d=