ef6811ef88
* Improved Java 17 support and Java runtime docs. 1) Add a "Java runtime" doc page with information about supported Java versions, garbage collection, and strong encapsulation.. 2) Update asm and equalsverifier to versions that support Java 17. 3) Add additional "--add-opens" lines to surefire configuration, so tests can pass successfully under Java 17. 4) Switch openjdk15 tests to openjdk17. 5) Update FrameFile to specifically mention Java runtime incompatibility as the cause of not being able to use Memory.map. 6) Update SegmentLoadDropHandler to log an error for Errors too, not just Exceptions. This is important because an IllegalAccessError is encountered when the correct "--add-opens" line is not provided, which would otherwise be silently ignored. 7) Update example configs to use druid.indexer.runner.javaOptsArray instead of druid.indexer.runner.javaOpts. (The latter is deprecated.) * Adjustments. * Use run-java in more places. * Add run-java. * Update .gitignore. * Exclude hadoop-client-api. Brought in when building on Java 17. * Swap one more usage of java. * Fix the run-java script. * Fix flag. * Include link to Temurin. * Spelling. * Update examples/bin/run-java Co-authored-by: Xavier Léauté <xl+github@xvrl.net> Co-authored-by: Xavier Léauté <xl+github@xvrl.net>
3.8 KiB
| id | title |
|---|---|
| build | Build from source |
You can build Apache Druid directly from source. Use the version of this page
that matches the version you want to build.
For building the latest code in master, follow the latest version of this page
here:
make sure it has /master/ in the URL.
Prerequisites
Installing Java and Maven
- JDK 8, 8u92+ or JDK 11. See our Java documentation for information about obtaining a JDK.
- Maven version 3.x
Other dependencies
- Distribution builds require Python 3.x and the
pyyamlmodule
Downloading the source
git clone git@github.com:apache/druid.git
cd druid
Building from source
The basic command to build Druid from source is:
mvn clean install
This will run static analysis, unit tests, compile classes, and package the projects into JARs. It will not generate the source or binary distribution tarball.
In addition to the basic stages, you may also want to add the following profiles and properties:
- -Pdist - Distribution profile: Generates the binary distribution tarball by pulling in core extensions and dependencies and packaging the files as
distribution/target/apache-druid-x.x.x-bin.tar.gz - -Papache-release - Apache release profile: Generates GPG signature and checksums, and builds the source distribution tarball as
distribution/target/apache-druid-x.x.x-src.tar.gz - -Prat - Apache Rat profile: Runs the Apache Rat license audit tool
- -DskipTests - Skips unit tests (which reduces build time)
- -Ddruid.console.skip=true - Skip front end project
Putting these together, if you wish to build the source and binary distributions with signatures and checksums, audit licenses, and skip the unit tests, you would run:
mvn clean install -Papache-release,dist,rat -DskipTests
Building hadoop 3 distribution
By default, druid ships hadoop 2.x.x jars along with the distribution. Exact version can be found in the main pom. To build druid with hadoop 3.x.x jars, hadoop3 profile needs to be activated.
To generate build with hadoop 3 dependencies, run:
mvn clean install -Phadoop3
To generate distribution with hadoop3 dependencies, run :
mvn clean install -Papache-release,dist-hadoop3,rat,hadoop3 -DskipTests
Potential issues
Missing pyyaml
You are building Druid from source following the instructions on this page but you get
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (generate-binary-license) on project distribution: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
Resolution: Make sure you have Python installed as well as the yaml module:
pip install pyyaml
On some systems, ensure you use the Python 3.x version of pip:
pip3 install pyyaml