Also upgrade our Dockerfile for release scripts to use ubuntu 22.04,
maven 3.8.6 and yetus 0.14.1.
Use python3 only.
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Exclude hbase-shaded-testing-util*.jar from checkcompatibility; this
jar can not be unzipped on a case-insensitive filesystem. Added
some means of debug into the checkcompatibility to help when
cryptic failures.
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Added logging of timestamp so we can tell where we are spending time.
Added context to the README copied from head of entrance script.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
README
Add note on ssh-agent.
dev-support/create-release/do-release.sh
move gpg check to non-docker context. Also cleanup tmp files when done.
dev-support/create-release/hbase-rm/Dockerfile
dev-support/create-release/mac-sshd-gpg-agent/Dockerfile
Hack to update packages... the old ones no longer available.
dev-support/create-release/release-util.sh
Allow that there are no JIRA changes in a release. Good for testing.
* put together a docker container that can use host gpg-agent forwarded over ssh.
* use gpg-agent forwarding container on OS X and directly forward the agent on Linux
* clean up the release container on exit
* use docker mounts instead of the deprecated volume syntax
* use image names within our project namespace
* update README to walk through running on GCE with gpg-agent forwarding
closes#1620
Signed-off-by: Matt Foley <mattf@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
By default, `urllib.quote` will skipp over `/` characters, which are
valid for use in passwords.
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Matt Foley <mattf@apache.org>
* adds a optional -r [repo] arg
* if the passed repo is on the local filesystem, creates a container mount
* when cloning a local repo configure git to share objects with the local repo instead of copying
* when cloning a local repo in a container configure the clone to have a remote that will work back on the host.
closes#1725
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
* non-zero exit for usage
* in non-docker mode prompt for yetus install location
* make sure we exit if YETUS_HOME does not actually point at a yetus install.
closes#1726
Signed-off-by: Matt Foley <mattf@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
* narrow 'dry-run' limits so see svn activity up to just before check-in.
* Fix several typos and, in case of dry run, enable build step to chain from tag step by keeping the tagged git repo.
* Improve Maven settings.xml file, and names of variables related to it. Remove unnecessary use of "-Dmaven.repo.local", put it in settings.xml instead. Stop putting password literals in settings.xml.
* stop deleting maven settings file, now that it doesn't contain password strings
* Merge 'tag' into release-build.sh as another stage. Also found and fixed a couple bugs in the current release-tag.sh.
* Delete redundant release-tag.sh script.
* Small changes to make dev-support/create-release tools less focused
on hbase project only, while retaining special behaviors for
hbase sub-projects which share the hbase dist and jira locations.
* Changed terminology of release publish steps, from <build|publish>
to <publish-dist|publish-release>. In fact, what was formerly called
"build" built the distribution tarballs AND published them to dist,
while "publish" built the maven artifacts AND published them to Nexus.
The new terminology clarifies what's happening, and removes the appearance
of order dependendency.
* Fix publish-snapshot so it does same checks as publish-release.
* Factor out common maven usages, and move them to build-util.sh.
* Change default polarity of DRY_RUN to default to true. Change -n ("no-publish") to -f ("force publish") to actually publish.
* Fix problems in do-release.sh so it runs correctly outside of docker,
including DRY_RUN being exported.
* Have do-release.sh set REPO (shared maven local repository) if doing all three stages.
* Cleaned up REPO directory creation.
* General cleanup of comments and usage.
* fix all 'shellcheck' errors
* use ${BASH_SOURCE[0]} instead of $0 to determine script directory path
* smarter way to read version from pom with mvn
* do maven-gpg-plugin config settings in maven settings file correctly as documented
* fix gpg signing failure on Mac due to gpg-agent timeout
* fix various bugs to enable publish-dist, publish-snapshot, and publish-release to work correctly as individual steps and/or without docker
* improve log reporting from publish-release step
* fix bug in argument to checkcompatibility.py: replace PACKAGE_VERSION with GIT_REF
* demote "PACKAGE_VERSION" to "package_version_name" and undocument it in favor of RELEASE_TAG. Still enable appropriate defaulting in case RELEASE_TAG is undefined.
* unify RELEASE_VERSION with VERSION, to remove ambiguity and allow it to be set when only running 'publish' step without 'tag'
* query confirm RELEASE_TAG
* emphasize that release-build.sh is called for a single action at a time, and should be called from do-release.sh.
* add '-s' option to do-release.sh
* suppress maven "Download from central:" messages
* Replace human wait for tag propagation
Signed-off-by: Cesar Delgado <cdelgado@apple.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Current code gives me the following; notice default values generated
for `RELEASE_VERSION` and `api_diff_tag`.
```
GIT_BRANCH [branch-2.3]:
Current branch VERSION is 2.3.0-SNAPSHOT.
RELEASE_VERSION [2.3.-1]: 2.3.0
NEXT_VERSION [2.3.0-SNAPSHOT]: 2.3.1-SNAPSHOT
RC_COUNT [0]:
GIT_REF [2.3.0RC0]:
api_diff_tag, [rel/2.2.0)]:
```
With this patch I get
```
GIT_BRANCH [branch-2.3]:
Current branch VERSION is 2.3.0-SNAPSHOT.
RELEASE_VERSION [2.3.0]:
NEXT_VERSION [2.3.1-SNAPSHOT]:
RC_COUNT [0]:
GIT_REF [2.3.0RC0]:
api_diff_tag, [rel/2.2.0]:
```
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Make the scripts generic. Adds an option that allows you specify
'project'. Defaults to 'hbase' for core. Pass 'hbase-thirdparty'
or 'hbase-operator-tools' etc.
This commit includes a bunch of bugfixes and miscellaneous
that came of trying to use the scripts making RCs.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
These scripts came originally from spark [1]. They were then
modified to suit hbase context. Supercedes the old
../make_rc.sh script because what is here is more comprehensive
doing more steps of the RM process as well as running in a
container so the RM build environment can be a constant.
It:
* Tags release
* Updates RELEASENOTES.md and CHANGES.md.
* Sets version to the release version
* Sets version to next SNAPSHOT version.
* Builds, signs, and hashes all artifacts.
* Generates the API report.
* Pushes release tgzs to the dev dir in a apache dist.
* Pushes to repository.apache.org staging.
* Generates a vote email with filled-in fields.
The entry point is the do-release-docker.sh script. Pass -h to
see available options. For example, running below will do all
steps above using the 'rm' dir under Downloads as workspace:
$ ./do-release-docker.sh -d ~/Downloads/rm
1. https://github.com/apache/spark/tree/master/dev/create-release
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Duo Zhang
Nick Dimiduk
Michael Stack
stack
Sean Busbey
Peter Somogyi
Matt Foley
Frederick Ayala