diff --git a/src/main/docbkx/developer.xml b/src/main/docbkx/developer.xml index 4d113c5a40e..49f99ca63e8 100644 --- a/src/main/docbkx/developer.xml +++ b/src/main/docbkx/developer.xml @@ -206,152 +206,12 @@ mvn clean package -DskipTests for examples
- Making A Release - I'll explain by running through a release. See later in this section for more detail on particular steps. - Update CHANGES.txt with the changes since the last release (query JIRA, export to excel then hack w/ vim to format to suit CHANGES.txt). - Adjust the version in all the poms appropriately; e.g. you may need to remove -SNAPSHOT from all versions. - The Versions Maven Plugin can be of use here. To - set a version in all poms, do something like this: - $ mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set -DnewVersion=0.96.0 - Checkin the CHANGES.txt and version changes. - - Now, build the src tarball. This tarball is hadoop version independent (it is pure src code). - $ MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease - Undo the tarball and make sure it looks good (A good test is seeing if you can build from the undone tarball). - Save it off to a version directory, e.g. hbase-0.96.0, the directory we will later publish as the release. - - Now we are into the hadoop1 or hadoop2 specific builds. Lets do hadoop1 first. - # Generate the hadoop1 poms. See the script's usage for what it expects by way of arguments. -$ ./dev-support/generate-hadoopX-poms.sh 0.96.0 0.96.0-hadoop1 -# Build the hadoop1 tarball. Note we include generation of javadoc and site and how we reference pom.xml.hadoop1 created by the above script run -$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop1 clean install -DskipTests javadoc:aggregate site assembly:single -Prelease -# Undo the tgz and check it out. Look at doc. and see if it runs, etc. Are the set of modules appropriate: -# e.g. do we have a hbase-hadoop2-compat in the hadoop1 tgz? -# Now deploy to mvn. Do mvn and tgz for a particular version all together in the one go else if flipping between hadoop1 and hadoop2, you might mal-publish poms and doctored hbase-default.xml to mvn repo. -# Use the apache-release profile instead of just 'release'; it will sign your artifacts published to mvn as long as your settings.xml in your local .m2 repository is configured correctly. -$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop1 deploy -DskipTests -Papache-release -# The above copies all artifacts for hadoop1 up to mvn repo. If no '-SNAPSHOT' in the version, it puts the artifacts into a staging directory. This is what you want. Later if the vote on release -# passes, you can promote them as released artifacts. - - - - Lets do the hadoop2 artifacts (read above hadoop1 section closely before coming here). - # Generate the hadoop2 poms. -$ ./dev-support/generate-hadoopX-poms.sh 0.96.0 0.96.0-hadoop2 -# Build the hadoop2 tarball. -$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop2 clean install -DskipTests javadoc:aggregate site assembly:single -Prelease -# Undo the tgz and check it out. -# Now deploy to mvn. -$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop2 deploy -DskipTests -Papache-release - - - - - - Making release tarballs -- src, hadoop1, and hadoop2 -- do as follows making use of the mvn - versions plugin to rewrite the hbase version string between builds to add the hadoop1 or haddop2 - suffix to the version as appropriate (Only the src tgz is built without modification from the tag - checkout; the binary builds will have unchecked-in modifications to their pom adding the - hadoop1/hadoop2 suffix to the version string). - $ MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease -# Copy away the src tgz found in hbase-assembly/target after verifying the content of the tgz is good. -# Now set the version in all poms to include -hadoop1 -$ mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set -DnewVersion=0.95.1-hadoop1 -# Create a tgz w/ this -hadoop1 appended to version. -$ MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/hadoop-one-compat.xml -Dhadoop.profile=1.0 -Prelease -# Copy away the hadoop1 bing tgz found in hbase-assembly/target after verifying the content of the tgz is good. -# Here is how to do a publish of a 'snapshot' up to mvn repo of this -hadoop1 version.VEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease -$ MAVEN_OPTS="-Xmx2g" mvn clean install deploy -DskipTests -Dhadoop.profile=1.0 -Prelease -# Now set the version to -hadoop2 -$ mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set -DnewVersion=0.95.1-hadoop2 -$ MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/hadoop-two-compat.xml -Dhadoop.profile=2.0 -Prelease -# Copy away the hadoop2 bing tgz found in hbase-assembly/target after verifying the content of the tgz is good. -# Here is how to do a publish of a 'snapshot' up to mvn repo of this -hadoop1 version. -$ MAVEN_OPTS="-Xmx2g" mvn clean install deploy -DskipTests -Dhadoop.profile=2.0 -Prelease - - - Next sign the tgzs and deploy (see the hadoop how to release instructions for how). -
- - - - - -
- Adding an Apache HBase release to Apache's Maven Repository - Follow the instructions at - Publishing Maven Artifacts after - reading the below miscellaney. - + Making Release Candidate + I'll explain by running through the process. See later in this section for more detail on particular steps. You must use maven 3.0.x (Check by running mvn -version). - Let me list out the commands I used first. The sections that follow dig in more - on what is going on. In this example, we are releasing the 0.92.2 jar to the apache - maven repository. - - # First make a copy of the tag we want to release; presumes the release has been tagged already - # We do this because we need to make some commits for the mvn release plugin to work. - 853 svn copy -m "Publishing 0.92.2 to mvn" https://svn.apache.org/repos/asf/hbase/tags/0.92.2 https://svn.apache.org/repos/asf/hbase/tags/0.92.2mvn - 857 svn checkout https://svn.apache.org/repos/asf/hbase/tags/0.92.2mvn - 858 cd 0.92.2mvn/ - # Edit the version making it release version with a '-SNAPSHOT' suffix (See below for more on this) - 860 vi pom.xml - 861 svn commit -m "Add SNAPSHOT to the version" pom.xml - 862 ~/bin/mvn/bin/mvn release:clean - 865 ~/bin/mvn/bin/mvn release:prepare - 866 # Answer questions and then ^C to kill the build after the last question. See below for more on this. - 867 vi release.properties - # Change the references to trunk svn to be 0.92.2mvn. The release plugin presumes trunk. - # Where it asks for a label for the release specify a label that does not yet exist; e.g. - # 0.92.2mvn2 (i.e. add a character, here a '2', to the label you are currently working - # against so when maven goes to label the release, it does not clash with an existing one). - # Then restart the release:prepare -- it won't ask questions because the properties file exists. - 868 ~/bin/mvn/bin/mvn release:prepare - # The apache-release profile comes from the apache parent pom and does signing of artifacts published - 869 ~/bin/mvn/bin/mvn release:perform -Papache-release - # When done copying up to apache staging repository, - # browse to repository.apache.org, login and finish - # the release as according to the above - # "Publishing Maven Artifacts. - - - Below is more detail on the commmands listed above. - At the mvn release:perform step, before starting, if you are for example - releasing hbase 0.92.2, you need to make sure the pom.xml version is 0.92.2-SNAPSHOT. This needs - to be checked in. Since we do the maven release after actual release, I've been doing this - checkin into a copy of the release tag rather than into the actual release tag itself (presumes the release has been properly tagged in svn). - So, say we released hbase 0.92.2 and now we want to do the release to the maven repository, in svn, the 0.92.2 - release will be tagged 0.92.2. Making the maven release, copy the 0.92.2 tag to 0.92.2mvn. - Check out this tag and change the version therein and commit. - - - Currently, the mvn release wants to go against trunk. I haven't figured how to tell it to do otherwise - so I do the below hack. The hack comprises answering the questions put to you by the mvn release plugin properly, - then immediately control-C'ing the build after the last question asked as the build release step starts to run. - After control-C'ing it, You'll notice a release.properties in your build dir. Review it. - Make sure it is using the proper branch -- it tends to use trunk rather than the 0.92.2mvn or whatever - that you want it to use -- so hand edit the release.properties file that was put under ${HBASE_HOME} - by the release:perform invocation. When done, resstart the - release:perform. - - Here is how I'd answer the questions at release:prepare time: - What is the release version for "HBase"? (org.apache.hbase:hbase) 0.92.2: : -What is SCM release tag or label for "HBase"? (org.apache.hbase:hbase) hbase-0.92.2: : 0.92.2mvn -What is the new development version for "HBase"? (org.apache.hbase:hbase) 0.92.3-SNAPSHOT: : -[INFO] Transforming 'HBase'... - - When you run release:perform, pass -Papache-release - else it will not 'sign' the artifacts it uploads. - - A strange issue I ran into was the one where the upload into the apache - repository was being sprayed across multiple apache machines making it so I could - not release. See INFRA-4482 Why is my upload to mvn spread across multiple repositories?. - - Here is my ~/.m2/settings.xml. - This is read by the release plugin. The apache-release profile will pick up your - gpg key setup from here if you've specified it into the file. The password - can be maven encrypted as suggested in the "Publishing Maven Artifacts" but plain - text password works too (just don't let anyone see your local settings.xml). + Make sure your local settings.xml is set up properly before you begin. + Here is my ~/.m2/settings.xml. <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 @@ -389,29 +249,84 @@ What is the new development version for "HBase"? (org.apache.hbase:hbase) 0.92.3 </settings> - - If you see run into the below, its because you need to edit version in the pom.xml and add - -SNAPSHOT to the version (and commit). - [INFO] Scanning for projects... -[INFO] Searching repository for plugin with prefix: 'release'. -[INFO] ------------------------------------------------------------------------ -[INFO] Building HBase -[INFO] task-segment: [release:prepare] (aggregator-style) -[INFO] ------------------------------------------------------------------------ -[INFO] [release:prepare {execution: default-cli}] -[INFO] ------------------------------------------------------------------------ -[ERROR] BUILD FAILURE -[INFO] ------------------------------------------------------------------------ -[INFO] You don't have a SNAPSHOT project in the reactor projects list. -[INFO] ------------------------------------------------------------------------ -[INFO] For more information, run Maven with the -e switch -[INFO] ------------------------------------------------------------------------ -[INFO] Total time: 3 seconds -[INFO] Finished at: Sat Mar 26 18:11:07 PDT 2011 -[INFO] Final Memory: 35M/423M -[INFO] ----------------------------------------------------------------------- + Update CHANGES.txt with the changes since the last release (query JIRA, export to excel then hack w/ vim to format to suit CHANGES.txt TODO: Needs detail). + Adjust the version in all the poms appropriately; e.g. you may need to remove -SNAPSHOT from all versions. + The Versions Maven Plugin can be of use here. To + set a version in all poms, do something like this: + $ mvn clean org.codehaus.mojo:versions-maven-plugin:1.3.1:set -DnewVersion=0.96.0 + Checkin the CHANGES.txt and version changes. + Now, build the src tarball. This tarball is hadoop version independent. It is just the pure src code and is without the hadoop1 or hadoop2 taint. + $ MAVEN_OPTS="-Xmx2g" mvn clean install javadoc:aggregate site assembly:single -DskipTests -Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease + Undo the tarball and make sure it looks good (A good test is seeing if you can build from the undone tarball). + Save it off to a version directory, i.e a directory somewhere where you are collecting + all of the tarballs you will publish as part of the release candidate. For example if we were building a + hbase-0.96.0 release candidate, we might call the directory hbase-0.96.0RC0. Later + we will publish this directory as our release candidate up on people.apache.org/~you. + + Now we are into the making of the hadoop1 and hadoop2 specific builds. Lets do hadoop1 first. + # First generate the hadoop1 poms. See the generate-hadoopX-poms.sh script usage for what it expects by way of arguments. +$ ./dev-support/generate-hadoopX-poms.sh 0.96.0 0.96.0-hadoop1 +# The script will work silently if all goes well. It will drop a pom.xml.hadoop1 beside all pom.xmls in all modules. +# Now build the hadoop1 tarball. Note we ask for the generation of javadoc and site and we reference the new pom.xml.hadoop1 explicitly +$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop1 clean install -DskipTests javadoc:aggregate site assembly:single -Prelease +# Undo the tgz and check it out. Look at doc. and see if it runs, etc. Are the set of modules appropriate: e.g. do we have a hbase-hadoop2-compat in the hadoop1 tgz? +# If good, copy the tarball to your 'version directory' +# Now deploy hadoop1 hbase to mvn. Do the mvn deploy and tgz for a particular version all together in the one go else if you flip between hadoop1 and hadoop2 builds, you might mal-publish poms and hbase-default.xml's (the version interpolations won't match). +# Use the apache-release profile instead of just 'release' when doing mvn deploy; it will sign your artifacts published to mvn as long as your settings.xml in your local .m2 repository is configured correctly. +$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop1 deploy -DskipTests -Papache-release -Pgpg +# The above copies all artifacts for hadoop1 up to mvn repo. The -Pgpg reads pgp password from settings.xml if it is set up properly. If no '-SNAPSHOT' in the version, it puts the artifacts into a staging directory. This is what you want. + + + + Lets do the hadoop2 artifacts (read above hadoop1 section closely before coming here because don't want to repeat detail since repeats have tendency to drift). + # Generate the hadoop2 poms. +$ ./dev-support/generate-hadoopX-poms.sh 0.96.0 0.96.0-hadoop2 +# Build the hadoop2 tarball. +$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop2 clean install -DskipTests javadoc:aggregate site assembly:single -Prelease +# Undo the tgz and check it out. If good, copy the tarball to your 'version directory'. Now deploy to mvn. +$ MAVEN_OPTS="-Xmx3g" mvn -f pom.xml.hadoop2 deploy -DskipTests -Papache-release -Pgpg + + + + At this stage we have three tarballs in our 'version directory' and two sets of artifacts up in maven in staging area. + First lets put the 'version directory' up on people.apache.org. You will need to sign and fingerprint them before you + push them up. In the 'version directory' do this: + $ for i in *.tar.gz; do echo $i; gpg --print-mds $i > $i.mds ; done +$ for i in *.tar.gz; do echo $i; gpg --armor --output $i.asc --detach-sig $i ; done +$ cd .. +# Presuming our 'version directory' is named 0.96.0RC0, now copy it up to people.apache.org. +$ rsync -av 0.96.0RC0 people.apache.org:public_html + + + + For the maven artifacts, login at repository.apache.org. Find your artifacts in the staging directory. + Close the artifacts. This will give you an URL for the temporary mvn staging repository. Do the closing + for hadoop1 and hadoop2 repos. + See Publishing Maven Artifacts for + some pointers. + + We no longer publish using the maven release plugin. Instead we do mvn deploy. It seems to give + us a backdoor to maven release publishing. If no -SNAPSHOT on the version + string, then we are 'deployed' to the apache maven repository staging directory from which we + can publish URLs for candidates and later, if they pass, publish as release (if a + -SNAPSHOT on the version string, deploy will put the artifacts up into + apache snapshot repos). + + + + Make sure the people.apache.org directory is showing -- it can take a while to show -- and that the + mvn repo urls are good. + Announce the release candidate on the mailing list and call a vote. + + A strange issue I ran into was the one where the upload into the apache + repository was being sprayed across multiple apache machines making it so I could + not release. See INFRA-4482 Why is my upload to mvn spread across multiple repositories?. + +
+ +
Generating the HBase Reference Guide The manual is marked up using docbook.