hbase/dev-support/create-release/README.txt

Creates an HBase release candidate.

The scripts in this directory came originally from spark
(https://github.com/apache/spark/tree/master/dev/create-release). They were
then modified to suit the hbase context. These scripts supercede the old
_../make_rc.sh_ script for making release candidates because what is here is
more comprehensive doing more steps of the RM process as well as running in a
container so the RM build environment can be a constant.

It:

 * Tags release
 * Sets version to the release version
 * Sets version to next SNAPSHOT version.
 * Builds, signs, and hashes all artifacts.
 * Pushes release tgzs to the dev dir in a apache dist.
 * Pushes to repository.apache.org staging.

The entry point is the do-release-docker.sh script. It requires a local
docker; for example, on mac os x, a Docker for Desktop installed and running.

(To run a build w/o invoking docker (not recommended!), use _do_release.sh_.)

The scripts will query interactively for needed parameters and passphrases.
For explanation of the parameters, execute:

 $ release-build.sh --help

The scripts run in dry-run mode by default where only local builds are
performed and nothing is uploaded to the ASF repos. Pass the '-f' flag
to remove dry-run mode.

Before starting the RC build, run a reconciliation of what is in JIRA with
what is in the commit log. Make sure they align and that anomalies are
explained up in JIRA. See http://hbase.apache.org/book.html#maven.release
for how.

Regardless of where your release build will run (locally, locally in docker,
on a remote machine, etc) you will need a local gpg-agent with access to your
secret keys. Before starting an RC build, make sure your local gpg-agent has
configs to properly handle your credentials, especially if you want to avoid
typing the passphrase to your secret key: e.g. if you are going to run
and step away (the RC creation takes ~5 hours), best to increase the TTL on
caching the unlocked secret by setting the following into local your
~/.gnupg/gpg-agent.conf file:
  # in seconds, e.g. a day
  default-cache-ttl 86400
  max-cache-ttl 86400

A quick way to tell gpg to clear out state, re-read the gpg-agent.conf file
and start a new gpg-agent is via the following command phrase:

 $ gpgconf --kill all && gpg-connect-agent /bye

You can verify options took hold with '$ gpg --list-options gpg-agent'.

Similarly, run ssh-agent with your ssh key added if building with docker.

Running a build on GCE is easy enough. Here are some notes if of use.
Create an instance. 4CPU/15G/20G disk seems to work well enough.
Once up, run the below to make your machine fit for RC building:

# Presuming debian-compatible OS, do these steps on the VM.
# Your VM username should be your ASF id, because it will show up in build artifacts.
# Follow the docker install guide: https://docs.docker.com/engine/install/debian/
$ sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg2 \
    software-properties-common
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
$ sudo add-apt-repository -y \
   "deb [arch=amd64] https://download.docker.com/linux/debian \
   $(lsb_release -cs) \
   stable"
$ sudo apt-get update
$ sudo apt-get install -y docker-ce docker-ce-cli containerd.io
# Follow the post installation steps: https://docs.docker.com/engine/install/linux-postinstall/
$ sudo usermod -aG docker $USER
# LOGOUT and then LOGIN again so $USERID shows as part of docker group
# Test here by running docker's hello world as your build user
$ docker run hello-world

# Follow the GPG guide for forwarding your gpg-agent from your local machine to the VM
#   https://wiki.gnupg.org/AgentForwarding
# On the VM find out the location of the gpg agent socket and extra socket
$ gpgconf --list-dir agent-socket
/run/user/1000/gnupg/S.gpg-agent
$ gpgconf --list-dir agent-extra-socket
/run/user/1000/gnupg/S.gpg-agent.extra
# On the VM configure sshd to remove stale sockets
$ sudo bash -c 'echo "StreamLocalBindUnlink yes" >> /etc/ssh/sshd_config'
$ sudo systemctl restart ssh
# logout of the VM

# Do these steps on your local machine.
# make sure gpg-agent is running
$ gpg-connect-agent /bye
# Export your public key and copy it to the VM.
# Assuming 'example.gce.host' maps to your VM's external IP (or use the IP)
$ gpg --export example@apache.org > ~/gpg.example.apache.pub
$ scp ~/gpg.example.apache.pub example.gce.host:
# ssh into the VM while forwarding the remote gpg socket locations found above to your local
#   gpg-agent's extra socket (this will restrict what commands the remote node is allowed to have
#   your agent handle. Note that the gpg guide above can help you set this up in your ssh config
#   rather than typing it in ssh like this every time.
# Note that as of maven-gpg-plugin 3.0.1, with gnupg >= 2.1, the plugin uses
#   `--pinentry-mode error`, which is apparently not supported over the `extra` socket. These
#   instructions may require tweaking.
$ ssh -i ~/.ssh/my_id \
    -R "/run/user/1000/gnupg/S.gpg-agent:$(gpgconf --list-dir agent-extra-socket)" \
    -R "/run/user/1000/gnupg/S.gpg-agent.extra:$(gpgconf --list-dir agent-extra-socket)" \
    example.gce.host

# now in an SSH session on the VM with the socket forwarding
# import your public key and test signing with the forwarding to your local agent.
$ gpg --no-autostart --import gpg.example.apache.pub
$ echo "foo" > foo.txt
$ gpg --no-autostart --detach --armor --sign foo.txt
$ gpg --no-autostart --verify foo.txt.asc

# install git and clone the main project on the build machine
$ sudo apt-get install -y git
$ git clone https://github.com/apache/hbase.git
# finally set up an output folder and launch a dry run.
$ mkdir ~/build
$ cd hbase
$ ./dev-support/create-release/do-release-docker.sh -d ~/build