hbase/dev-support/create-release/README.txt

52 lines
2.0 KiB
Plaintext
Raw Normal View History

Entrance script is _do-release-docker.sh_. Requires a local docker;
for example, on mac os x, Docker for Desktop installed and running.
For usage, pass '-h':
$ ./do-release-docker.sh -h
HBASE-24318 Create-release scripts fixes and enhancements (#1643) * narrow 'dry-run' limits so see svn activity up to just before check-in. * Fix several typos and, in case of dry run, enable build step to chain from tag step by keeping the tagged git repo. * Improve Maven settings.xml file, and names of variables related to it. Remove unnecessary use of "-Dmaven.repo.local", put it in settings.xml instead. Stop putting password literals in settings.xml. * stop deleting maven settings file, now that it doesn't contain password strings * Merge 'tag' into release-build.sh as another stage. Also found and fixed a couple bugs in the current release-tag.sh. * Delete redundant release-tag.sh script. * Small changes to make dev-support/create-release tools less focused on hbase project only, while retaining special behaviors for hbase sub-projects which share the hbase dist and jira locations. * Changed terminology of release publish steps, from <build|publish> to <publish-dist|publish-release>. In fact, what was formerly called "build" built the distribution tarballs AND published them to dist, while "publish" built the maven artifacts AND published them to Nexus. The new terminology clarifies what's happening, and removes the appearance of order dependendency. * Fix publish-snapshot so it does same checks as publish-release. * Factor out common maven usages, and move them to build-util.sh. * Change default polarity of DRY_RUN to default to true. Change -n ("no-publish") to -f ("force publish") to actually publish. * Fix problems in do-release.sh so it runs correctly outside of docker, including DRY_RUN being exported. * Have do-release.sh set REPO (shared maven local repository) if doing all three stages. * Cleaned up REPO directory creation. * General cleanup of comments and usage. * fix all 'shellcheck' errors * use ${BASH_SOURCE[0]} instead of $0 to determine script directory path * smarter way to read version from pom with mvn * do maven-gpg-plugin config settings in maven settings file correctly as documented * fix gpg signing failure on Mac due to gpg-agent timeout * fix various bugs to enable publish-dist, publish-snapshot, and publish-release to work correctly as individual steps and/or without docker * improve log reporting from publish-release step * fix bug in argument to checkcompatibility.py: replace PACKAGE_VERSION with GIT_REF * demote "PACKAGE_VERSION" to "package_version_name" and undocument it in favor of RELEASE_TAG. Still enable appropriate defaulting in case RELEASE_TAG is undefined. * unify RELEASE_VERSION with VERSION, to remove ambiguity and allow it to be set when only running 'publish' step without 'tag' * query confirm RELEASE_TAG * emphasize that release-build.sh is called for a single action at a time, and should be called from do-release.sh. * add '-s' option to do-release.sh * suppress maven "Download from central:" messages * Replace human wait for tag propagation Signed-off-by: Cesar Delgado <cdelgado@apple.com> Signed-off-by: stack <stack@apache.org> Signed-off-by: Sean Busbey <busbey@apache.org>
2020-05-09 11:46:41 -04:00
To run a build w/o invoking docker (not recommended!), use _do_release.sh_.
Both scripts will query interactively for needed parameters and passphrases.
For explanation of the parameters, execute:
$ release-build.sh --help
Before starting the RC build, run a reconciliation of what is in
JIRA with what is in the commit log. Make sure they align and that
anomalies are explained up in JIRA.
See http://hbase.apache.org/book.html#maven.release
Running a build on GCE is easy enough. Here are some notes if of use.
Create an instance. 4CPU/15G/10G disk seems to work well enough.
Once up, run the below to make your machine fit for RC building:
# Presuming debian-compatible OS
$ sudo apt-get install -y git openjdk-8-jdk maven gnupg gnupg-agent
# Install docker
$ sudo apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
$ sudo add-apt-repository -y \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
$ sudo apt-get update
$ sudo apt-get install -y docker-ce docker-ce-cli containerd.io
$ sudo usermod -a -G docker $USERID
HBASE-24318 Create-release scripts fixes and enhancements (#1643) * narrow 'dry-run' limits so see svn activity up to just before check-in. * Fix several typos and, in case of dry run, enable build step to chain from tag step by keeping the tagged git repo. * Improve Maven settings.xml file, and names of variables related to it. Remove unnecessary use of "-Dmaven.repo.local", put it in settings.xml instead. Stop putting password literals in settings.xml. * stop deleting maven settings file, now that it doesn't contain password strings * Merge 'tag' into release-build.sh as another stage. Also found and fixed a couple bugs in the current release-tag.sh. * Delete redundant release-tag.sh script. * Small changes to make dev-support/create-release tools less focused on hbase project only, while retaining special behaviors for hbase sub-projects which share the hbase dist and jira locations. * Changed terminology of release publish steps, from <build|publish> to <publish-dist|publish-release>. In fact, what was formerly called "build" built the distribution tarballs AND published them to dist, while "publish" built the maven artifacts AND published them to Nexus. The new terminology clarifies what's happening, and removes the appearance of order dependendency. * Fix publish-snapshot so it does same checks as publish-release. * Factor out common maven usages, and move them to build-util.sh. * Change default polarity of DRY_RUN to default to true. Change -n ("no-publish") to -f ("force publish") to actually publish. * Fix problems in do-release.sh so it runs correctly outside of docker, including DRY_RUN being exported. * Have do-release.sh set REPO (shared maven local repository) if doing all three stages. * Cleaned up REPO directory creation. * General cleanup of comments and usage. * fix all 'shellcheck' errors * use ${BASH_SOURCE[0]} instead of $0 to determine script directory path * smarter way to read version from pom with mvn * do maven-gpg-plugin config settings in maven settings file correctly as documented * fix gpg signing failure on Mac due to gpg-agent timeout * fix various bugs to enable publish-dist, publish-snapshot, and publish-release to work correctly as individual steps and/or without docker * improve log reporting from publish-release step * fix bug in argument to checkcompatibility.py: replace PACKAGE_VERSION with GIT_REF * demote "PACKAGE_VERSION" to "package_version_name" and undocument it in favor of RELEASE_TAG. Still enable appropriate defaulting in case RELEASE_TAG is undefined. * unify RELEASE_VERSION with VERSION, to remove ambiguity and allow it to be set when only running 'publish' step without 'tag' * query confirm RELEASE_TAG * emphasize that release-build.sh is called for a single action at a time, and should be called from do-release.sh. * add '-s' option to do-release.sh * suppress maven "Download from central:" messages * Replace human wait for tag propagation Signed-off-by: Cesar Delgado <cdelgado@apple.com> Signed-off-by: stack <stack@apache.org> Signed-off-by: Sean Busbey <busbey@apache.org>
2020-05-09 11:46:41 -04:00
# LOGOUT and then LOGIN again so $USERID shows as part of docker group
# Copy up private key for $USERID export from laptop and import on gce.
$ gpg --import stack.duboce.net.asc
$ export GPG_TTY=$(tty) # https://github.com/keybase/keybase-issues/issues/2798
$ eval $(gpg-agent --disable-scdaemon --daemon --no-grab --allow-preset-passphrase --default-cache-ttl=86400 --max-cache-ttl=86400)
HBASE-24318 Create-release scripts fixes and enhancements (#1643) * narrow 'dry-run' limits so see svn activity up to just before check-in. * Fix several typos and, in case of dry run, enable build step to chain from tag step by keeping the tagged git repo. * Improve Maven settings.xml file, and names of variables related to it. Remove unnecessary use of "-Dmaven.repo.local", put it in settings.xml instead. Stop putting password literals in settings.xml. * stop deleting maven settings file, now that it doesn't contain password strings * Merge 'tag' into release-build.sh as another stage. Also found and fixed a couple bugs in the current release-tag.sh. * Delete redundant release-tag.sh script. * Small changes to make dev-support/create-release tools less focused on hbase project only, while retaining special behaviors for hbase sub-projects which share the hbase dist and jira locations. * Changed terminology of release publish steps, from <build|publish> to <publish-dist|publish-release>. In fact, what was formerly called "build" built the distribution tarballs AND published them to dist, while "publish" built the maven artifacts AND published them to Nexus. The new terminology clarifies what's happening, and removes the appearance of order dependendency. * Fix publish-snapshot so it does same checks as publish-release. * Factor out common maven usages, and move them to build-util.sh. * Change default polarity of DRY_RUN to default to true. Change -n ("no-publish") to -f ("force publish") to actually publish. * Fix problems in do-release.sh so it runs correctly outside of docker, including DRY_RUN being exported. * Have do-release.sh set REPO (shared maven local repository) if doing all three stages. * Cleaned up REPO directory creation. * General cleanup of comments and usage. * fix all 'shellcheck' errors * use ${BASH_SOURCE[0]} instead of $0 to determine script directory path * smarter way to read version from pom with mvn * do maven-gpg-plugin config settings in maven settings file correctly as documented * fix gpg signing failure on Mac due to gpg-agent timeout * fix various bugs to enable publish-dist, publish-snapshot, and publish-release to work correctly as individual steps and/or without docker * improve log reporting from publish-release step * fix bug in argument to checkcompatibility.py: replace PACKAGE_VERSION with GIT_REF * demote "PACKAGE_VERSION" to "package_version_name" and undocument it in favor of RELEASE_TAG. Still enable appropriate defaulting in case RELEASE_TAG is undefined. * unify RELEASE_VERSION with VERSION, to remove ambiguity and allow it to be set when only running 'publish' step without 'tag' * query confirm RELEASE_TAG * emphasize that release-build.sh is called for a single action at a time, and should be called from do-release.sh. * add '-s' option to do-release.sh * suppress maven "Download from central:" messages * Replace human wait for tag propagation Signed-off-by: Cesar Delgado <cdelgado@apple.com> Signed-off-by: stack <stack@apache.org> Signed-off-by: Sean Busbey <busbey@apache.org>
2020-05-09 11:46:41 -04:00
$ export PROJECT="${PROJECT:-hbase}"
$ git clone https://github.com/apache/${PROJECT}.git
$ cd "${PROJECT}"
$ mkdir ~/build
$ ./dev-resources/create-release/do-release-docker.sh -d ~/build
# etc.