Commit Graph

467 Commits

Author SHA1 Message Date
Nick Dimiduk 5bada0b5d0
HBASE-22853 Git/Jira Release Audit Tool (#1088)
* HBASE-22853 Git/Jira Release Audit Tool

This is an application for performing an audit between the histories
on our git branches and the `fixVersion` field set on issues in
JIRA. It does this by building a Sqlite database from the commits
found on each git branch, identifying Jira IDs and release tags, and
then requesting information about those issues from Jira. Once both
sources have been collected, queries can be performed against the
database to look for discrepancies between the sources of truth (and,
possibly, bugs in this script).

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-27 10:28:11 -08:00
stack a58f2a4a55 HBASE-23720 [create-release] Update yetus version used from 0.11.0 to 0.11.1 2020-01-22 18:07:53 -08:00
Josh Elser 04d789f1a7 HBASE-23347 Allow custom authentication methods for RPCs
Decouple the HBase internals such that someone can implement
their own SASL-based authentication mechanism and plug it into
HBase RegionServers/Masters.

Comes with a design doc in dev-support/design-docs and an example in
hbase-examples known as "Shade" which uses a flat-password file
for authenticating users.

Closes #884

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-01-16 10:27:28 -05:00
Beata Sudi de87d72bb4 HBASE-22920 github pr testing job should use dev-support script (#883)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2019-12-13 15:56:20 -08:00
Sean Busbey 97e0107000
HBASE-23337 Release scripts should rely on maven for deploy. (#887)
- switch to nexus-staging-maven-plugin for asf-release
- refactor release-build to use mvn deploy and its output.
- cleaned up some tabs in the root pom

Signed-off-by: stack <stack@apache.org>
2019-12-02 06:39:24 -06:00
Duo Zhang 4b99816dd6
HBASE-23182 The create-release scripts are broken (#736)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2019-11-19 08:28:43 +08:00
Sean Busbey d5dcc090c1
HBASE-23228 Allow for jdk8 specific modules on branch-1 in precommit/nightly testing (#804)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-11-08 10:07:24 -06:00
Duo Zhang b53e9a325c HBASE-23236 Upgrade to yetus 0.11.1
Signed-off-by: stack <stack@apache.org>
2019-11-08 13:47:54 +08:00
Peter Somogyi fd9cfd789b
HBASE-23114 Use archiveArtifacts in Jenkinsfiles (#681)
Signed-off-by: Michael Stack <stack@apache.org>
2019-10-10 13:27:50 +02:00
Michael Stack 57439e5bf5
HBASE-23137 [create-release] Add passing of PROJECT variable for when building other than core (#701) 2019-10-08 15:12:14 -07:00
Frederick Ayala 95c9911a9c HBASE-23031 Upgrade Yetus version in RM scripts (#692)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-06 11:24:35 +02:00
stack f69e7705e0 HBASE-23121 create-release is not pushing artifacts to repository.a.o 2019-10-04 15:32:17 -07:00
Duo Zhang f08a310ed9
HBASE-23118 [RELEASE SCRIPTS] Allow creating a RC from an existing tag[RELEASE SCRIPTS] Allow creating a RC from an existing tag (#686)
Signed-off-by: stack <stack@apache.org>
2019-10-04 13:39:27 +08:00
Michael Stack 7ee6d59ef8
HBASE-23092 Make the RM tooling in dev-tools/create-release generic (#671)
Make the scripts generic. Adds an option that allows you specify
'project'. Defaults to 'hbase' for core. Pass 'hbase-thirdparty'
or 'hbase-operator-tools' etc.

This commit includes a bunch of bugfixes and miscellaneous
that came of trying to use the scripts making RCs.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-02 09:14:17 -07:00
Peter Somogyi 56489819bc
HBASE-23053 Disable concurrent nightly builds (#654)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-23 15:00:22 +02:00
Sean Busbey bcad0d9f98 HBASE-22955 Properly set jdk7 as JAVA_HOME for branches-1 precommit and nightly. (#621)
master/branches-2 specific changes: work around yetus overwriting JAVA_HOME
in the container with the host JAVA_HOME.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 41990ba20a)
2019-09-16 22:51:24 -05:00
Sean Busbey 1cb4f687c1
HBASE-23023 upgrade shellcheck used in dockerfile (#624)
Signed-off-by: stack <stack@apache.org>
2019-09-14 14:03:02 -05:00
Peter Somogyi 51b6ce391b
HBASE-22981 Remove unused flags for Yetus (#591)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-06 14:56:05 +02:00
Duo Zhang dfe5280d0c HBASE-22724 Addendum trim trailing spaces 2019-09-04 09:03:58 +08:00
Duo Zhang da144888d0
HBASE-22724 Add a emoji on the vote table for pre commit result on github (#568)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Stack <stack@apache.org>
2019-09-04 08:52:28 +08:00
Rushabh 0438fdc0f6 HBASE-22852 Stop gpg agent daemons in hbase nightlies. (#525)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-08-26 16:30:22 -05:00
Gav 89e54f2899 HBASE-22913 Use Hadoop label for nightly builds (#532)
HBase, a Hadoop related project, must use the Hadoop label please.
This build, and others are starving the 'ubuntu' label which other projects need to use.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-24 16:51:26 +08:00
Sean Busbey 94155fe175 HBASE-22911 reduce concurrent PR builds.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-24 16:30:36 +08:00
meiyi 3b16ae2720 HBASE-22625 documet use scan snapshot feature (#496)
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-08-23 15:04:53 -07:00
Artem Ervits 74fb2040ea HBASE-22464 Improvements to hbase-vote script
* wget should use timestamps to avoid re-downloading RC artifacts we already have.
* allow changing to maven profiles other than runAllTests for test step

Signed-off-by: Sean Busbey <busbey@apache.org>
2019-08-23 12:44:58 -05:00
Duo Zhang 288d1f2769
HBASE-22891 Use HBaseQA in HBase-PreCommit-GitHub-PR job (#518)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-08-22 23:07:05 +08:00
Duo Zhang d9d5f69fc6 HBASE-22856 HBASE-Find-Flaky-Tests fails with pip error 2019-08-15 10:14:08 +08:00
stack 53db390f60 HBASE-22854 Remove +x shell spew from do-release-*.sh create-release scripts 2019-08-14 10:05:41 -07:00
Peter Somogyi 9d8271624e HBASE-22801 Maven build issue on Github PRs 2019-08-06 18:57:14 +02:00
Duo Zhang c9293b0b12 HBASE-22719 Add debug support for github PR pre commit job 2019-07-22 13:49:27 +08:00
stack ef44f30006 HBASE-21953 Point make_rc.sh at ./create-release/do-release-docker.sh instead 2019-06-12 07:54:30 -07:00
stack 8745d5e7e8 HBASE-21935 Replace make_rc.sh with customized spark/dev/create-release
These scripts came originally from spark [1]. They were then
modified to suit hbase context. Supercedes the old
../make_rc.sh script because what is here is more comprehensive
doing more steps of the RM process as well as running in a
container so the RM build environment can be a constant.

It:
 * Tags release
 * Updates RELEASENOTES.md and CHANGES.md.
 * Sets version to the release version
 * Sets version to next SNAPSHOT version.
 * Builds, signs, and hashes all artifacts.
 * Generates the API report.
 * Pushes release tgzs to the dev dir in a apache dist.
 * Pushes to repository.apache.org staging.
 * Generates a vote email with filled-in fields.

The entry point is the do-release-docker.sh script. Pass -h to
see available options. For example, running below will do all
steps above using the 'rm' dir under Downloads as workspace:

 $ ./do-release-docker.sh  -d ~/Downloads/rm

1. https://github.com/apache/spark/tree/master/dev/create-release

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-06-12 07:53:49 -07:00
Josh Elser 2f9d995163 HBASE-22563 Reduce old builds by half
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-06-10 18:49:16 -04:00
Sean Busbey b4d5a9a8e1 HBASE-22518 yetus personality is treating branch-1.4 like earlier branches for hadoopcheck
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-06-03 09:16:34 -05:00
Sean Busbey 51c49a1784 HBASE-22497 Mark HBase 1.2 line as EOM
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-06-03 06:55:58 -05:00
Guanghao Zhang ab99e2486e HBASE-22522 The integration test in master branch's nightly job has error "ERROR: Only found 1050 rows." 2019-06-02 22:07:49 +08:00
Guanghao Zhang 12584ebf46 HBASE-22490 (addendum) Nightly client integration test fails with hadoop-3 2019-05-31 14:46:16 +08:00
Duo Zhang e4f4ae835f HBASE-22500 Modify pom and jenkins jobs for hadoop versions
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-05-30 22:31:32 +08:00
Guanghao 70fb30cd56
HBASE-22490 Nightly client integration test fails with hadoop-3.0.3 2019-05-30 10:29:40 +08:00
zhangduo e919ca6ca7 HBASE-22471 Our nightly jobs for master and branch-2 are still using hadoop-2.7.1 in integration test 2019-05-28 22:57:50 +08:00
Peter Somogyi 83b8472c9a HBASE-22474 Add --mvn-custom-repo parameter to yetus calls 2019-05-26 18:11:22 +02:00
Guanghao Zhang de579cd243 HBASE-22462 (addendum) Should run a 'mvn install' at the end of hadoop check in pre commit job 2019-05-24 09:23:20 +08:00
Guanghao Zhang 6b899ccbbc HBASE-22462 Should run a 'mvn install' at the end of hadoop check in pre commit job 2019-05-23 20:57:23 +08:00
zhangduo b74df9b747 HBASE-22442 Nightly build is failing with hadoop 3.x 2019-05-19 07:54:29 +08:00
Duo Zhang 5a578f953b HBASE-20970 Update hadoop check versions for hadoop3 in hbase-personality
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-05-17 10:01:51 +08:00
Andrew Purtell e1d023d54c
HBASE-22430 hbase-vote should tee build and test output to console 2019-05-16 16:45:53 -07:00
Tak Lon (Stephen) Wu 25a1c5a619 HBASE-22429 hbase-vote download step requires URL to end with '/' (#243) 2019-05-16 16:43:27 -07:00
Duo Zhang 083605df8f HBASE-22399 Change default hadoop-two.version to 2.8.x and remove the 2.7.x hadoop checks 2019-05-14 16:45:20 +08:00
Sean Busbey 60dc5143f2 HBASE-22406 skip generating rdoc when building gems in our docker image for running yetus
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-13 11:21:16 -05:00
Sean Busbey b5a6fd14fa HBASE-22020 update nightly to yetus 0.9.0
Signed-off-by: stack <stack@apache.org>
2019-04-24 07:31:45 -05:00