HBASE-20356 Make skipping protoc possible

This commit is contained in:
Mike Drob 2018-04-11 21:25:04 -05:00
parent 0c751dadf8
commit a50d9f4351
3 changed files with 29 additions and 8 deletions

View File

@ -554,7 +554,7 @@ function hbaseprotoc_rebuild
# Need to run 'install' instead of 'compile' because shading plugin
# is hooked-up to 'install'; else hbase-protocol-shaded is left with
# half of its process done.
modules_workers patch hbaseprotoc install -DskipTests -Pcompile-protobuf -X -DHBasePatchProcess
modules_workers patch hbaseprotoc install -DskipTests -X -DHBasePatchProcess
# shellcheck disable=SC2153
until [[ $i -eq "${#MODULE[@]}" ]]; do

View File

@ -106,7 +106,7 @@
<version>1.5.3</version>
<executions>
<execution>
<phase>generate-sources</phase>
<phase>process-sources</phase>
<goals>
<goal>replace</goal>
</goals>
@ -117,6 +117,9 @@
<includes>
<include>**/*.java</include>
</includes>
<!-- Ignore errors when missing files, because it means this build
was run with -Dprotoc.skip and there is no -Dreplacer.skip -->
<ignoreErrors>true</ignoreErrors>
<replacements>
<replacement>
<token>([^\.])com.google.protobuf</token>

View File

@ -415,22 +415,40 @@ use so we can freely change versions without upsetting any downstream project us
The protobuf files are located in _hbase-protocol/src/main/protobuf_.
For the change to be effective, you will need to regenerate the classes.
You can use maven profile `compile-protobuf` to do this.
[source,bourne]
----
mvn compile -Pcompile-protobuf
mvn package -pl hbase-protocol -am
----
You may also want to define `protoc.path` for the protoc binary, using the following command:
Similarly, protobuf definitions for internal use are located in the _hbase-protocol-shaded_ module.
[source,bourne]
----
mvn compile -Pcompile-protobuf -Dprotoc.path=/opt/local/bin/protoc
mvn package -pl hbase-protocol-shaded -am
----
Read the _hbase-protocol/README.txt_ for more details.
Typically, protobuf code generation is done using the native `protoc` binary. In our build we use a maven plugin for
convenience; however, the plugin may not be able to retrieve appropriate binaries for all platforms. If you find yourself
on a platform where protoc fails, you will have to compile protoc from source, and run it independent of our maven build.
You can disable the inline code generation by specifying `-Dprotoc.skip` in your maven arguments, allowing your build to proceed further.
A similar failure relates to the stock CentOS 6 docker image providing a too old version of glibc for the version of protoc that we use.
In this case, you would have to install glibc 2.14 and protoc 3.5.1 manually, then execute something like:
[source,bourne]
----
cd hbase-protocol-shaded
LD_LIBRARY_PATH=/opt/glibc-2.14/lib protoc \
--proto_path=src/main/protobuf \
--java_out=target/generated-sources/protobuf/java \
src/main/protobuf/*.proto
----
[NOTE]
If you need to manually generate your protobuf files, you should not use `clean` in subsequent maven calls, as that will delete the newly generated files.
Read the _hbase-protocol/README.txt_ for more details
[[build.thrift]]
==== Build Thrift