HADOOP-15045. ISA-L bulid options are documented in branch-2.

This commit is contained in:
Akira Ajisaka 2017-11-21 13:50:49 +09:00
parent ed879d4a72
commit 4eb4f37bcf
No known key found for this signature in database
GPG Key ID: C1EDBB9CA400FD50
1 changed files with 17 additions and 42 deletions

View File

@ -4,7 +4,7 @@ Build instructions for Hadoop
Requirements: Requirements:
* Unix System * Unix System
* JDK 1.8+ * JDK 1.7 or 1.8
* Maven 3.0 or later * Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs) * Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0 * ProtocolBuffer 2.5.0
@ -14,7 +14,6 @@ Requirements:
* Linux FUSE (Filesystem in Userspace) version 2.6 or above (if compiling fuse_dfs) * Linux FUSE (Filesystem in Userspace) version 2.6 or above (if compiling fuse_dfs)
* Internet connection for first build (to fetch all Maven and Hadoop dependencies) * Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* python (for releasedocs) * python (for releasedocs)
* bats (for shell code testing)
* Node.js / bower / Ember-cli (for YARN UI v2 building) * Node.js / bower / Ember-cli (for YARN UI v2 building)
---------------------------------------------------------------------------------- ----------------------------------------------------------------------------------
@ -57,12 +56,12 @@ Known issues:
---------------------------------------------------------------------------------- ----------------------------------------------------------------------------------
Installing required packages for clean install of Ubuntu 14.04 LTS Desktop: Installing required packages for clean install of Ubuntu 14.04 LTS Desktop:
* Oracle JDK 1.8 (preferred) * Oracle JDK 1.7 (preferred)
$ sudo apt-get purge openjdk* $ sudo apt-get purge openjdk*
$ sudo apt-get install software-properties-common $ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:webupd8team/java $ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update $ sudo apt-get update
$ sudo apt-get install oracle-java8-installer $ sudo apt-get install oracle-java7-installer
* Maven * Maven
$ sudo apt-get -y install maven $ sudo apt-get -y install maven
* Native libraries * Native libraries
@ -74,9 +73,6 @@ Optional packages:
* Snappy compression * Snappy compression
$ sudo apt-get install snappy libsnappy-dev $ sudo apt-get install snappy libsnappy-dev
* Intel ISA-L library for erasure coding
Please refer to https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version
(OR https://github.com/01org/isa-l)
* Bzip2 * Bzip2
$ sudo apt-get install bzip2 libbz2-dev $ sudo apt-get install bzip2 libbz2-dev
* Jansson (C Library for JSON) * Jansson (C Library for JSON)
@ -111,7 +107,7 @@ Maven build goals:
* Clean : mvn clean [-Preleasedocs] * Clean : mvn clean [-Preleasedocs]
* Compile : mvn compile [-Pnative] * Compile : mvn compile [-Pnative]
* Run tests : mvn test [-Pnative] [-Pshelltest] * Run tests : mvn test [-Pnative]
* Create JAR : mvn package * Create JAR : mvn package
* Run findbugs : mvn compile findbugs:findbugs * Run findbugs : mvn compile findbugs:findbugs
* Run checkstyle : mvn compile checkstyle:checkstyle * Run checkstyle : mvn compile checkstyle:checkstyle
@ -189,38 +185,6 @@ Maven build goals:
For example, to run test_bulk_crc32, you would use: For example, to run test_bulk_crc32, you would use:
mvn test -Pnative -Dtest=test_bulk_crc32 mvn test -Pnative -Dtest=test_bulk_crc32
Intel ISA-L build options:
Intel ISA-L is an erasure coding library that can be utilized by the native code.
It is currently an optional component, meaning that Hadoop can be built with
or without this dependency. Note the library is used via dynamic module. Please
reference the official site for the library details.
https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version
(OR https://github.com/01org/isa-l)
* Use -Drequire.isal to fail the build if libisal.so is not found.
If this option is not specified and the isal library is missing,
we silently build a version of libhadoop.so that cannot make use of ISA-L and
the native raw erasure coders.
This option is recommended if you plan on making use of native raw erasure
coders and want to get more repeatable builds.
* Use -Disal.prefix to specify a nonstandard location for the libisal
library files. You do not need this option if you have installed ISA-L to the
system library path.
* Use -Disal.lib to specify a nonstandard location for the libisal library
files.
* Use -Dbundle.isal to copy the contents of the isal.lib directory into
the final tar file. This option requires that -Disal.lib is also given,
and it ignores the -Disal.prefix option. If -Disal.lib isn't given, the
bundling and building will fail.
Special plugins: OWASP's dependency-check:
OWASP's dependency-check plugin will scan the third party dependencies
of this project for known CVEs (security vulnerabilities against them).
It will produce a report in target/dependency-check-report.html. To
invoke, run 'mvn dependency-check:aggregate'. Note that this plugin
requires maven 3.1.1 or greater.
---------------------------------------------------------------------------------- ----------------------------------------------------------------------------------
Building components separately Building components separately
@ -316,8 +280,8 @@ Building on Windows
Requirements: Requirements:
* Windows System * Windows System
* JDK 1.8+ * JDK 1.7 or 1.8
* Maven 3.3 or later * Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs) * Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0 * ProtocolBuffer 2.5.0
* CMake 2.6 or newer * CMake 2.6 or newer
@ -391,3 +355,14 @@ http://www.zlib.net/
Building distributions: Building distributions:
* Build distribution with native code : mvn package [-Pdist][-Pdocs][-Psrc][-Dtar] * Build distribution with native code : mvn package [-Pdist][-Pdocs][-Psrc][-Dtar]
----------------------------------------------------------------------------------
Running compatibility checks with checkcompatibility.py
Invoke `./dev-support/bin/checkcompatibility.py` to run Java API Compliance Checker
to compare the public Java APIs of two git objects. This can be used by release
managers to compare the compatibility of a previous and current release.
As an example, this invocation will check the compatibility of interfaces annotated as Public or LimitedPrivate:
./dev-support/bin/checkcompatibility.py --annotation org.apache.hadoop.classification.InterfaceAudience.Public --annotation org.apache.hadoop.classification.InterfaceAudience.LimitedPrivate --include "hadoop.*" branch-2.7.2 trunk