Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

First cut at doc on how to stage a release using new means -- the pom generating scripts and the mvn door I recently discovered 

git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1519124 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Michael Stack 2013-08-31 00:27:51 +00:00
parent 5a3060ee37
commit c41e90e54b
1 changed files with 85 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
src/main/docbkx/developer.xml
View File

@ -32,7 +32,9 @@
</para>
<section xml:id="repos">
<title>Apache HBase Repositories</title>
<para>There are two different repositories for Apache HBase: Subversion (SVN) and Git. The former is the system of record for committers, but the latter is easier to work with to build and contribute. SVN updates get automatically propagated to the Git repo.</para>
<para>There are two different repositories for Apache HBase: Subversion (SVN) and Git. The former
is the system of record for committers, but the latter is easier to work with to build and contribute.
SVN updates get automatically propagated to the Git repo.</para>
<section xml:id="svn">
<title>SVN</title>
<programlisting>
@ -149,7 +151,8 @@ Access restriction: The method getLong(Object, long) from the type Unsafe is not
<title>Building Apache HBase</title>
<section xml:id="build.basic">
<title>Basic Compile</title>
<para>Thanks to maven, building HBase is pretty easy. You can read about the various maven commands in <xref linkend="maven.build.commands"/>, but the simplest command to compile HBase from its java source code is:
<para>Thanks to maven, building HBase is pretty easy. You can read about the various maven commands in <xref linkend="maven.build.commands"/>,
but the simplest command to compile HBase from its java source code is:
<programlisting>
mvn package -DskipTests
</programlisting>
@ -157,7 +160,22 @@ mvn package -DskipTests
<programlisting>
mvn clean package -DskipTests
</programlisting>
With Eclipse set up as explained above in <xref linkend="eclipse"/>, you can also simply use the build command in Eclipse. To create the full installable HBase package takes a little bit more work, so read on.
With Eclipse set up as explained above in <xref linkend="eclipse"/>, you can also simply use the build command in Eclipse.
To create the full installable HBase package takes a little bit more work, so read on.
</para>
</section>
<section xml:id="build.protobuf"><title>Build Protobuf</title>
<para>You may need to change the protobuf definitions that reside in the hbase-protocol module.</para>
<para>
The protobuf files are located in <link xlink:href="https://svn.apache.org/repos/asf/hbase/trunk/hbase-protocol/src/main/protobuf">hbase-protocol/src/main/protobuf</link>.
For the change to be effective, you will need to
regenerate the classes (read the <link xlink:href="https://svn.apache.org/repos/asf/hbase/trunk/hbase-protocol/README.txt">hbase-protocol/README.txt</link> for more details).
</para>
</section>
<section xml:id="build.gotchas"><title>Build Gotchas</title>
<para>If you see <code>Unable to find resource 'VM_global_library.vm'</code>, ignore it.
Its not an error. It is <link xlink:href="http://jira.codehaus.org/browse/MSITE-286">officially ugly</link> though.
</para>
</section>
<section xml:id="build.snappy">
@ -165,15 +183,71 @@ mvn clean package -DskipTests
<para>Pass <code>-Dsnappy</code> to trigger the snappy maven profile for building
snappy native libs into hbase. See also <xref linkend="snappy.compression" /></para>
</section>
</section>
<section xml:id="build.tgz">
<title>Building the HBase tarball</title>
<para>Do the following to build the HBase tarball.
Passing the -Prelease will generate javadoc and run the RAT plugin to verify licenses on source.
<programlisting>% MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Prelease</programlisting>
<section xml:id="releasing">
<title>Releasing Apache HBase</title>
<para>HBase 0.96.x will run on hadoop 1.x or hadoop 2.x but building, you
must choose which to build against; we cannot make a single HBase binary
to run against both hadoop1 and hadoop2. Since we include the Hadoop we were built
against -- so we can do standalone mode -- the set of modules included
in the tarball changes dependent on whether the hadoop1 or hadoop2 target chosen.
You can tell which HBase you have -- whether it is for hadoop1 or hadoop2
by looking at the version; the HBase for hadoop1 will include 'hadoop1' in its
version. Ditto for hadoop2.
</para>
<para>Maven, our build system, natively will not let you have a single product
built against different dependencies. Its understandable. But neither could
we convince maven to change the set of included modules and write out
the correct poms w/ appropriate dependencies even though we have two
build targets; one for hadoop1 and another for hadoop2. So, there is a prestep
required. This prestep takes as input the current pom.xmls and it generates hadoop1 or
hadoop2 versions. You then reference these generated poms when you build. Read on
for examples</para>
<section xml:id="release">
<title>Making Release Tarballs</title>
<title>Making A Release</title>
<para>I'll explain by running through a release. See later in this section for more detail on particular steps.</para>
<para>Update CHANGES.txt with the changes since the last release (query JIRA, export to excel then hack w/ vim to format to suit CHANGES.txt).
Adjust the version in all the poms appropriately; e.g. you may need to remove <emphasis>-SNAPSHOT</emphasis> from all versions.
The <link xlink:href="http://mojo.codehaus.org/versions-maven-plugin/">Versions Maven Plugin</link> can be of use here. To
set a version in all poms, do something like this:
<programlisting>$ mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set -DnewVersion=0.96.0</programlisting>
Checkin the <filename>CHANGES.txt</filename> and version changes.
</para>
<para>Now, build the src tarball. This tarball is hadoop version independent (it is pure src code).
<programlisting>$ MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease</programlisting>
Undo the tarball and make sure it looks good (A good test is seeing if you can build from the undone tarball).
Save it off to a version directory, e.g. hbase-0.96.0, the directory we will later publish as the release.
</para>
<para>Now we are into the hadoop1 or hadoop2 specific builds. Lets do hadoop1 first.
<programlisting># Generate the hadoop1 poms. See the script's usage for what it expects by way of arguments.
$ ./dev-support/generate-hadoopX-poms.sh 0.96.0 0.96.0-hadoop1
# Build the hadoop1 tarball. Note we include generation of javadoc and site and how we reference pom.xml.hadoop1 created by the above script run
$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop1 clean install -DskipTests javadoc:aggregate site assembly:single -Prelease
# Undo the tgz and check it out. Look at doc. and see if it runs, etc. Are the set of modules appropriate:
# e.g. do we have a hbase-hadoop2-compat in the hadoop1 tgz?
# Now deploy to mvn. Do mvn and tgz for a particular version all together in the one go else if flipping between hadoop1 and hadoop2, you might mal-publish poms and doctored hbase-default.xml to mvn repo.
# Use the apache-release profile instead of just 'release'; it will sign your artifacts published to mvn as long as your settings.xml in your local .m2 repository is configured correctly.
$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop1 deploy -DskipTests -Papache-release
# The above copies all artifacts for hadoop1 up to mvn repo. If no '-SNAPSHOT' in the version, it puts the artifacts into a staging directory. This is what you want. Later if the vote on release
# passes, you can promote them as released artifacts.
</programlisting>
</para>
<para>Lets do the hadoop2 artifacts (read above hadoop1 section closely before coming here).
<programlisting># Generate the hadoop2 poms.
$ ./dev-support/generate-hadoopX-poms.sh 0.96.0 0.96.0-hadoop2
# Build the hadoop2 tarball.
$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop2 clean install -DskipTests javadoc:aggregate site assembly:single -Prelease
# Undo the tgz and check it out.
# Now deploy to mvn.
$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop2 deploy -DskipTests -Papache-release
</programlisting>
</para>
<para>Making release tarballs -- src, hadoop1, and hadoop2 -- do as follows making use of the mvn
versions plugin to rewrite the hbase version string between builds to add the hadoop1 or haddop2
suffix to the version as appropriate (Only the src tgz is built without modification from the tag
@ -186,7 +260,7 @@ $ mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set -DnewVersion=0.95.
# Create a tgz w/ this -hadoop1 appended to version.
$ MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/hadoop-one-compat.xml -Dhadoop.profile=1.0 -Prelease
# Copy away the hadoop1 bing tgz found in hbase-assembly/target after verifying the content of the tgz is good.
# Here is how to do a publish of a 'snapshot' up to mvn repo of this -hadoop1 version.
# Here is how to do a publish of a 'snapshot' up to mvn repo of this -hadoop1 version.VEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease
$ MAVEN_OPTS="-Xmx2g" mvn clean install deploy -DskipTests -Dhadoop.profile=1.0 -Prelease
# Now set the version to -hadoop2
$ mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set -DnewVersion=0.95.1-hadoop2
@ -202,21 +276,7 @@ $ MAVEN_OPTS="-Xmx2g" mvn clean install deploy -DskipTests -Dhadoop.profile=2.0
</section>
<section xml:id="build.protobuf"><title>Build Protobuf</title>
<para>You may need to change the protobuf definitions that reside in the hbase-protocol module.</para>
<para>
The protobuf files are located in It is <link xlink:href="https://svn.apache.org/repos/asf/hbase/trunk/hbase-protocol/src/main/protobuf">hbase-protocol/src/main/protobuf</link>.
For the change to be effective, you will need to regenerate the classes (read the <link xlink:href="https://svn.apache.org/repos/asf/hbase/trunk/hbase-protocol/README.txt">hbase-protocol/README.txt</link> for more details).
</para>
</section>
<section xml:id="build.gotchas"><title>Build Gotchas</title>
<para>If you see <code>Unable to find resource 'VM_global_library.vm'</code>, ignore it.
Its not an error. It is <link xlink:href="http://jira.codehaus.org/browse/MSITE-286">officially ugly</link> though.
</para>
</section>
</section> <!-- build -->
<section xml:id="mvn_repo">
<title>Adding an Apache HBase release to Apache's Maven Repository</title>
<para>Follow the instructions at