Creates an HBase release candidate. The scripts in this directory came originally from spark (https://github.com/apache/spark/tree/master/dev/create-release). They were then modified to suit the hbase context. These scripts supercede the old _../make_rc.sh_ script for making release candidates because what is here is more comprehensive doing more steps of the RM process as well as running in a container so the RM build environment can be a constant. It: * Tags release * Sets version to the release version * Sets version to next SNAPSHOT version. * Builds, signs, and hashes all artifacts. * Pushes release tgzs to the dev dir in a apache dist. * Pushes to repository.apache.org staging. The entry point is the do-release-docker.sh script. It requires a local docker; for example, on mac os x, a Docker for Desktop installed and running. (To run a build w/o invoking docker (not recommended!), use _do_release.sh_.) The scripts will query interactively for needed parameters and passphrases. For explanation of the parameters, execute: $ release-build.sh --help The scripts run in dry-run mode by default where only local builds are performed and nothing is uploaded to the ASF repos. Pass the '-f' flag to remove dry-run mode. Before starting the RC build, run a reconciliation of what is in JIRA with what is in the commit log. Make sure they align and that anomalies are explained up in JIRA. See http://hbase.apache.org/book.html#maven.release for how. Regardless of where your release build will run (locally, locally in docker, on a remote machine, etc) you will need a local gpg-agent with access to your secret keys. Before starting an RC build, make sure your local gpg-agent has configs to properly handle your credentials, especially if you want to avoid typing the passphrase to your secret key: e.g. if you are going to run and step away (the RC creation takes ~5 hours), best to increase the TTL on caching the unlocked secret by setting the following into local your ~/.gnupg/gpg-agent.conf file: # in seconds, e.g. a day default-cache-ttl 86400 max-cache-ttl 86400 A quick way to tell gpg to clear out state, re-read the gpg-agent.conf file and start a new gpg-agent is via the following command phrase: $ gpgconf --kill all && gpg-connect-agent /bye You can verify options took hold with '$ gpg --list-options gpg-agent'. Similarly, run ssh-agent with your ssh key added if building with docker. Running a build on GCE is easy enough. Here are some notes if of use. Create an instance. 4CPU/15G/20G disk seems to work well enough. Once up, run the below to make your machine fit for RC building: # Presuming debian-compatible OS, do these steps on the VM. # Your VM username should be your ASF id, because it will show up in build artifacts. # Follow the docker install guide: https://docs.docker.com/engine/install/debian/ $ sudo apt-get install -y \ apt-transport-https \ ca-certificates \ curl \ gnupg2 \ software-properties-common $ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add - $ sudo add-apt-repository -y \ "deb [arch=amd64] https://download.docker.com/linux/debian \ $(lsb_release -cs) \ stable" $ sudo apt-get update $ sudo apt-get install -y docker-ce docker-ce-cli containerd.io # Follow the post installation steps: https://docs.docker.com/engine/install/linux-postinstall/ $ sudo usermod -aG docker $USER # LOGOUT and then LOGIN again so $USERID shows as part of docker group # Test here by running docker's hello world as your build user $ docker run hello-world # Follow the GPG guide for forwarding your gpg-agent from your local machine to the VM # https://wiki.gnupg.org/AgentForwarding # On the VM find out the location of the gpg agent socket and extra socket $ gpgconf --list-dir agent-socket /run/user/1000/gnupg/S.gpg-agent $ gpgconf --list-dir agent-extra-socket /run/user/1000/gnupg/S.gpg-agent.extra # On the VM configure sshd to remove stale sockets $ sudo bash -c 'echo "StreamLocalBindUnlink yes" >> /etc/ssh/sshd_config' $ sudo systemctl restart ssh # logout of the VM # Do these steps on your local machine. # make sure gpg-agent is running $ gpg-connect-agent /bye # Export your public key and copy it to the VM. # Assuming 'example.gce.host' maps to your VM's external IP (or use the IP) $ gpg --export example@apache.org > ~/gpg.example.apache.pub $ scp ~/gpg.example.apache.pub example.gce.host: # ssh into the VM while forwarding the remote gpg socket locations found above to your local # gpg-agent's extra socket (this will restrict what commands the remote node is allowed to have # your agent handle. Note that the gpg guide above can help you set this up in your ssh config # rather than typing it in ssh like this every time. # Note that as of maven-gpg-plugin 3.0.1, with gnupg >= 2.1, the plugin uses # `--pinentry-mode error`, which is apparently not supported over the `extra` socket. These # instructions may require tweaking. $ ssh -i ~/.ssh/my_id \ -R "/run/user/1000/gnupg/S.gpg-agent:$(gpgconf --list-dir agent-extra-socket)" \ -R "/run/user/1000/gnupg/S.gpg-agent.extra:$(gpgconf --list-dir agent-extra-socket)" \ example.gce.host # now in an SSH session on the VM with the socket forwarding # import your public key and test signing with the forwarding to your local agent. $ gpg --no-autostart --import gpg.example.apache.pub $ echo "foo" > foo.txt $ gpg --no-autostart --detach --armor --sign foo.txt $ gpg --no-autostart --verify foo.txt.asc # install git and clone the main project on the build machine $ sudo apt-get install -y git $ git clone https://github.com/apache/hbase.git # finally set up an output folder and launch a dry run. $ mkdir ~/build $ cd hbase $ ./dev-support/create-release/do-release-docker.sh -d ~/build