Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-10020 Add maven compile-protobuf profile 

git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1546237 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Enis Soztutar 2013-11-27 23:57:23 +00:00
parent 12f0dce3ae
commit 770e336ef6
17 changed files with 1150 additions and 668 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
dev-support/build-proto.sh
View File

@ -1,36 +0,0 @@
#!/bin/bash
##
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
##
# script to run protoc to generate protocol buf files.
# usage: ./build-proto.sh
#
which protoc
if [ $? != 0 ] ; then
echo "Must have protoc compiler in your path to generate code"
exit 1
fi
HBASE_DIR=`dirname $0`/..
PROTO_DIR=$HBASE_DIR/hbase-protocol/src/main/protobuf
JAVA_DIR=$HBASE_DIR/hbase-protocol/src/main/java
set -x
for f in $PROTO_DIR/*.proto ; do
protoc -I$PROTO_DIR --java_out=$JAVA_DIR $f
done

41
hbase-examples/pom.xml
View File

@ -245,5 +245,44 @@ if we can combine these profiles somehow -->
</plugins>
</build>
</profile>
</profiles>
<profile>
<id>compile-protobuf</id>
<activation>
<property>
<name>compile-protobuf</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-maven-plugins</artifactId>
<executions>
<execution>
<id>compile-protoc</id>
<phase>generate-sources</phase>
<goals>
<goal>protoc</goal>
</goals>
<configuration>
<imports>
<param>${basedir}/src/main/protobuf</param>
<param>${basedir}/../hbase-protocol/src/main/protobuf</param>
</imports>
<source>
<directory>${basedir}/src/main/protobuf</directory>
<includes>
<include>BulkDelete.proto</include>
<include>Examples.proto</include>
</includes>
</source>
<output>${basedir}/src/main/java/</output>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

865
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/generated/BulkDeleteProtos.java
View File

File diff suppressed because it is too large Load Diff

584
hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/generated/ExampleProtos.java
View File

File diff suppressed because it is too large Load Diff

33
hbase-protocol/README.txt
View File

@ -7,28 +7,21 @@ the protobuf protoc tool is in your $PATH (You may need to download it and build
it first; its part of the protobuf package obtainable from here:
http://code.google.com/p/protobuf/downloads/list).
Then run the following (You should be able to just copy and paste the below into a
terminal and hit return -- the protoc compiler runs fast):
HBase uses hadoop-maven-plugins:protoc goal to invoke the protoc command. You can
compile the protoc definitions by invoking maven with profile compile-protobuf or
passing in compile-protobuf property.
UNIX_PROTO_DIR=src/main/protobuf
JAVA_DIR=src/main/java/
mkdir -p $JAVA_DIR 2> /dev/null
if which cygpath 2> /dev/null; then
PROTO_DIR=`cygpath --windows $UNIX_PROTO_DIR`
JAVA_DIR=`cygpath --windows $JAVA_DIR`
else
PROTO_DIR=$UNIX_PROTO_DIR
fi
# uncomment the next line if you want to remove before generating
# rm -fr $JAVA_DIR/org/apache/hadoop/hbase/protobuf/generated
for PROTO_FILE in $UNIX_PROTO_DIR/*.proto
do
protoc -I$PROTO_DIR --java_out=$JAVA_DIR $PROTO_FILE
done
mvn compile -Dcompile-protobuf
or
mvn compile -Pcompile-protobuf
You may also want to define protoc.path for the protoc binary
mvn compile -Dcompile-protobuf -Dprotoc.path=/opt/local/bin/protoc
If you have added a new proto file, you should add it to the pom.xml file first.
Other modules also support the maven profile.
After you've done the above, check it in and then check it in (or post a patch
on a JIRA with your definition file changes and the generated files).
Optionally, you can uncomment the hadoop-maven-plugins plugin in hbase-protocol/pom.xml.
This plugin will generate for the classes during the build. Once again, you will need protocol buffers
to be installed on your build machine (https://developers.google.com/protocol-buffers)

120
hbase-protocol/pom.xml
View File

@ -69,63 +69,6 @@
</execution>
</executions>
</plugin>
<!--
<plugin>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-maven-plugins</artifactId>
<version>2.0.5-alpha</version>
<executions>
<execution>
<id>compile-protoc</id>
<phase>generate-sources</phase>
<goals>
<goal>protoc</goal>
</goals>
<configuration>
<protocVersion>${protobuf.version}</protocVersion>
<protocCommand>${protoc.path}</protocCommand>
<imports>
<param>${basedir}/src/main/protobuf</param>
</imports>
<source>
<directory>${basedir}/src/main/protobuf</directory>
<includes>
<include>AccessControl.proto</include>
<include>Admin.proto</include>
<include>Aggregate.proto</include>
<include>Authentication.proto</include>
<include>Cell.proto</include>
<include>Client.proto</include>
<include>ClusterId.proto</include>
<include>ClusterStatus.proto</include>
<include>Comparator.proto</include>
<include>ErrorHandling.proto</include>
<include>Filter.proto</include>
<include>FS.proto</include>
<include>HBase.proto</include>
<include>HFile.proto</include>
<include>LoadBalancer.proto</include>
<include>MapReduce.proto</include>
<include>MasterAdmin.proto</include>
<include>MasterMonitor.proto</include>
<include>Master.proto</include>
<include>MultiRowMutationProcessorMessages.proto</include>
<include>MultiRowMutation.proto</include>
<include>RegionServerStatus.proto</include>
<include>RowProcessor.proto</include>
<include>RPC.proto</include>
<include>SecureBulkLoad.proto</include>
<include>Tracing.proto</include>
<include>WAL.proto</include>
<include>ZooKeeper.proto</include>
</includes>
</source>
<output>${project.build.directory}/generated-sources/java</output>
</configuration>
</execution>
</executions>
</plugin>
-->
</plugins>
<pluginManagement>
<plugins>
@ -180,6 +123,67 @@
<surefire.skipFirstPart>true</surefire.skipFirstPart>
</properties>
</profile>
<profile>
<id>compile-protobuf</id>
<activation>
<property>
<name>compile-protobuf</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-maven-plugins</artifactId>
<executions>
<execution>
<id>compile-protoc</id>
<phase>generate-sources</phase>
<goals>
<goal>protoc</goal>
</goals>
<configuration>
<imports>
<param>${basedir}/src/main/protobuf</param>
</imports>
<source>
<directory>${basedir}/src/main/protobuf</directory>
<includes>
<include>AccessControl.proto</include>
<include>Admin.proto</include>
<include>Aggregate.proto</include>
<include>Authentication.proto</include>
<include>Cell.proto</include>
<include>Client.proto</include>
<include>ClusterId.proto</include>
<include>ClusterStatus.proto</include>
<include>Comparator.proto</include>
<include>ErrorHandling.proto</include>
<include>Filter.proto</include>
<include>FS.proto</include>
<include>HBase.proto</include>
<include>HFile.proto</include>
<include>LoadBalancer.proto</include>
<include>MapReduce.proto</include>
<include>Master.proto</include>
<include>MultiRowMutation.proto</include>
<include>RegionServerStatus.proto</include>
<include>RowProcessor.proto</include>
<include>RPC.proto</include>
<include>SecureBulkLoad.proto</include>
<include>Tracing.proto</include>
<include>WAL.proto</include>
<include>ZooKeeper.proto</include>
</includes>
</source>
<!--<output>${project.build.directory}/generated-sources/java</output>-->
<output>${basedir}/src/main/java/</output>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

70
hbase-server/pom.xml
View File

@ -685,5 +685,75 @@
</plugins>
</build>
</profile>
<profile>
<id>compile-protobuf</id>
<activation>
<property>
<name>compile-protobuf</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-maven-plugins</artifactId>
<executions>
<execution>
<id>compile-protoc</id>
<phase>generate-sources</phase>
<goals>
<goal>protoc</goal>
</goals>
<configuration>
<imports>
<param>${basedir}/src/main/resources/org/apache/hadoop/hbase/rest/protobuf</param>
</imports>
<source>
<!-- These should be under src/main/protobuf -->
<directory>${basedir}/src/main/resources/org/apache/hadoop/hbase/rest/protobuf</directory>
<includes>
<include>CellMessage.proto</include>
<include>CellSetMessage.proto</include>
<include>ColumnSchemaMessage.proto</include>
<include>ScannerMessage.proto</include>
<include>StorageClusterStatusMessage.proto</include>
<include>TableInfoMessage.proto</include>
<include>TableListMessage.proto</include>
<include>TableSchemaMessage.proto</include>
<include>VersionMessage.proto</include>
</includes>
</source>
<output>${basedir}/src/main/java/</output>
</configuration>
</execution>
<execution>
<id>compile-test-protoc</id>
<phase>generate-test-sources</phase>
<goals>
<goal>protoc</goal>
</goals>
<configuration>
<imports>
<param>${basedir}/src/test/protobuf</param>
</imports>
<source>
<directory>${basedir}/src/test/protobuf</directory>
<includes>
<include>ColumnAggregationProtocol.proto</include>
<include>IncrementCounterProcessor.proto</include>
<include>PingProtocol.proto</include>
<include>test.proto</include>
<include>test_delayed_rpc.proto</include>
<include>test_rpc_service.proto</include>
</includes>
</source>
<output>${basedir}/src/test/java/</output>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

8
hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ColumnSchemaMessage.java
View File

@ -728,7 +728,7 @@ public final class ColumnSchemaMessage {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;
@ -802,7 +802,7 @@ public final class ColumnSchemaMessage {
getValueBytes() {
java.lang.Object ref = value_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
value_ = b;
@ -1394,7 +1394,7 @@ public final class ColumnSchemaMessage {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;
@ -1790,7 +1790,7 @@ public final class ColumnSchemaMessage {
getCompressionBytes() {
java.lang.Object ref = compression_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
compression_ = b;

2
hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ScannerMessage.java
View File

@ -1113,7 +1113,7 @@ public final class ScannerMessage {
getFilterBytes() {
java.lang.Object ref = filter_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
filter_ = b;

2
hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/StorageClusterStatusMessage.java
View File

@ -2405,7 +2405,7 @@ public final class StorageClusterStatusMessage {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;

6
hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableInfoMessage.java
View File

@ -806,7 +806,7 @@ public final class TableInfoMessage {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;
@ -985,7 +985,7 @@ public final class TableInfoMessage {
getLocationBytes() {
java.lang.Object ref = location_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
location_ = b;
@ -1449,7 +1449,7 @@ public final class TableInfoMessage {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;

6
hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableSchemaMessage.java
View File

@ -744,7 +744,7 @@ public final class TableSchemaMessage {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;
@ -818,7 +818,7 @@ public final class TableSchemaMessage {
getValueBytes() {
java.lang.Object ref = value_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
value_ = b;
@ -1446,7 +1446,7 @@ public final class TableSchemaMessage {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;

10
hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/VersionMessage.java
View File

@ -751,7 +751,7 @@ public final class VersionMessage {
getRestVersionBytes() {
java.lang.Object ref = restVersion_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
restVersion_ = b;
@ -825,7 +825,7 @@ public final class VersionMessage {
getJvmVersionBytes() {
java.lang.Object ref = jvmVersion_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
jvmVersion_ = b;
@ -899,7 +899,7 @@ public final class VersionMessage {
getOsVersionBytes() {
java.lang.Object ref = osVersion_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
osVersion_ = b;
@ -973,7 +973,7 @@ public final class VersionMessage {
getServerVersionBytes() {
java.lang.Object ref = serverVersion_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
serverVersion_ = b;
@ -1047,7 +1047,7 @@ public final class VersionMessage {
getJerseyVersionBytes() {
java.lang.Object ref = jerseyVersion_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
jerseyVersion_ = b;

6
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/protobuf/generated/PingProtos.java
View File

@ -807,7 +807,7 @@ public final class PingProtos {
getPongBytes() {
java.lang.Object ref = pong_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
pong_ = b;
@ -2980,7 +2980,7 @@ public final class PingProtos {
getNameBytes() {
java.lang.Object ref = name_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
name_ = b;
@ -3489,7 +3489,7 @@ public final class PingProtos {
getResponseBytes() {
java.lang.Object ref = response_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
response_ = b;

4
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/protobuf/generated/TestProtos.java
View File

@ -1145,7 +1145,7 @@ public final class TestProtos {
getMessageBytes() {
java.lang.Object ref = message_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
message_ = b;
@ -1662,7 +1662,7 @@ public final class TestProtos {
getMessageBytes() {
java.lang.Object ref = message_;
if (ref instanceof String) {
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString b =
com.google.protobuf.ByteString.copyFromUtf8(
(java.lang.String) ref);
message_ = b;

9
pom.xml
View File

@ -690,6 +690,15 @@
<attach>false</attach>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-maven-plugins</artifactId>
<version>${hadoop-two.version}</version>
<configuration>
<protocVersion>${protobuf.version}</protocVersion>
<protocCommand>${protoc.path}</protocCommand>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>

16
src/main/docbkx/developer.xml
View File

@ -165,11 +165,21 @@ mvn clean package -DskipTests
</para>
</section>
<section xml:id="build.protobuf"><title>Build Protobuf</title>
<para>You may need to change the protobuf definitions that reside in the hbase-protocol module.</para>
<para>You may need to change the protobuf definitions that reside in the hbase-protocol module or other modules.</para>
<para>
The protobuf files are located in <link xlink:href="https://svn.apache.org/repos/asf/hbase/trunk/hbase-protocol/src/main/protobuf">hbase-protocol/src/main/protobuf</link>.
For the change to be effective, you will need to
regenerate the classes (read the <link xlink:href="https://svn.apache.org/repos/asf/hbase/trunk/hbase-protocol/README.txt">hbase-protocol/README.txt</link> for more details).
For the change to be effective, you will need to regenerate the classes. You can use maven profile compile-protobuf to do this.
<programlisting>
mvn compile -Dcompile-protobuf
or
mvn compile -Pcompile-protobuf
</programlisting>
You may also want to define protoc.path for the protoc binary
<programlisting>
mvn compile -Dcompile-protobuf -Dprotoc.path=/opt/local/bin/protoc
</programlisting>
Read the <link xlink:href="https://svn.apache.org/repos/asf/hbase/trunk/hbase-protocol/README.txt">hbase-protocol/README.txt</link> for more details.
</para>
</section>