Creates an HBase release candidate.
The scripts in this directory came originally from spark
(https://github.com/apache/spark/tree/master/dev/create-release). They were
then modified to suit the hbase context. These scripts supercede the old
_../make_rc.sh_ script for making release candidates because what is here is
more comprehensive doing more steps of the RM process as well as running in a
container so the RM build environment can be a constant.
It:
* Tags release
* Sets version to the release version
* Sets version to next SNAPSHOT version.
* Builds, signs, and hashes all artifacts.
* Pushes release tgzs to the dev dir in a apache dist.
* Pushes to repository.apache.org staging.
The entry point is the do-release-docker.sh script. It requires a local
docker; for example, on mac os x, a Docker for Desktop installed and running.
(To run a build w/o invoking docker (not recommended!), use _do_release.sh_.)
The scripts will query interactively for needed parameters and passphrases.
For explanation of the parameters, execute:
$ release-build.sh --help
The scripts run in dry-run mode by default where only local builds are
performed and nothing is uploaded to the ASF repos. Pass the '-f' flag
to remove dry-run mode.
Before starting the RC build, run a reconciliation of what is in JIRA with
what is in the commit log. Make sure they align and that anomalies are
explained up in JIRA. See http://hbase.apache.org/book.html#maven.release
for how.
Regardless of where your release build will run (locally, locally in docker,
on a remote machine, etc) you will need a local gpg-agent with access to your
secret keys. Before starting an RC build, make sure your local gpg-agent has
configs to properly handle your credentials, especially if you want to avoid
typing the passphrase to your secret key: e.g. if you are going to run
and step away (the RC creation takes ~5 hours), best to increase the TTL on
caching the unlocked secret by setting the following into local your
~/.gnupg/gpg-agent.conf file:
# in seconds, e.g. a day
default-cache-ttl 86400
max-cache-ttl 86400
A quick way to tell gpg to clear out state, re-read the gpg-agent.conf file
and start a new gpg-agent is via the following command phrase:
$ gpgconf --kill all && gpg-connect-agent /bye
You can verify options took hold with '$ gpg --list-options gpg-agent'.
Similarly, run ssh-agent with your ssh key added if building with docker.
Running a build on GCE is easy enough. Here are some notes if of use.
Create an instance. 4CPU/15G/20G disk seems to work well enough.
Once up, run the below to make your machine fit for RC building:
# Presuming debian-compatible OS, do these steps on the VM.
# Your VM username should be your ASF id, because it will show up in build artifacts.
# Follow the docker install guide: https://docs.docker.com/engine/install/debian/
$ sudo apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
$ sudo add-apt-repository -y \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
$ sudo apt-get update
$ sudo apt-get install -y docker-ce docker-ce-cli containerd.io
# Follow the post installation steps: https://docs.docker.com/engine/install/linux-postinstall/
$ sudo usermod -aG docker $USER
# LOGOUT and then LOGIN again so $USERID shows as part of docker group
# Test here by running docker's hello world as your build user
$ docker run hello-world
# Follow the GPG guide for forwarding your gpg-agent from your local machine to the VM
# https://wiki.gnupg.org/AgentForwarding
# On the VM find out the location of the gpg agent socket and extra socket
$ gpgconf --list-dir agent-socket
/run/user/1000/gnupg/S.gpg-agent
$ gpgconf --list-dir agent-extra-socket
/run/user/1000/gnupg/S.gpg-agent.extra
# On the VM configure sshd to remove stale sockets
$ sudo bash -c 'echo "StreamLocalBindUnlink yes" >> /etc/ssh/sshd_config'
$ sudo systemctl restart ssh
# logout of the VM
# Do these steps on your local machine.
# make sure gpg-agent is running
$ gpg-connect-agent /bye
# Export your public key and copy it to the VM.
# Assuming 'example.gce.host' maps to your VM's external IP (or use the IP)
$ gpg --export example@apache.org > ~/gpg.example.apache.pub
$ scp ~/gpg.example.apache.pub example.gce.host:
# ssh into the VM while forwarding the remote gpg socket locations found above to your local
# gpg-agent's extra socket (this will restrict what commands the remote node is allowed to have
# your agent handle. Note that the gpg guide above can help you set this up in your ssh config
# rather than typing it in ssh like this every time.
$ ssh -i ~/.ssh/my_id \
-R "/run/user/1000/gnupg/S.gpg-agent:$(gpgconf --list-dir agent-extra-socket)" \
-R "/run/user/1000/gnupg/S.gpg-agent.extra:$(gpgconf --list-dir agent-extra-socket)" \
example.gce.host
# now in an SSH session on the VM with the socket forwarding
# import your public key and test signing with the forwarding to your local agent.
$ gpg --no-autostart --import gpg.example.apache.pub
$ echo "foo" > foo.txt
$ gpg --no-autostart --detach --armor --sign foo.txt
$ gpg --no-autostart --verify foo.txt.asc
# install git and clone the main project on the build machine
$ sudo apt-get install -y git
$ git clone https://github.com/apache/hbase.git
# finally set up an output folder and launch a dry run.
$ mkdir ~/build
$ cd hbase
$ ./dev-support/create-release/do-release-docker.sh -d ~/build
679ba95d7d