Commit Graph

376 Commits

Author SHA1 Message Date
Mike Drob b04c976fe6 HBASE-20478 Update checkstyle to v8.2
Cannot go to latest (8.9) yet due to
  https://github.com/checkstyle/checkstyle/issues/5279

* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors

Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-18 14:02:40 -07:00
Sean Busbey a4cb98b1cd HBASE-20733 QABot should run checkstyle tests if the checkstyle configs change
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-14 20:26:42 -05:00
Mike Drob 0a6aec4981 HBASE-19377 Update Java API CC version
Compatibility checker complaining about hash collisions, newer versions
use longer id strings.
2018-06-12 15:13:35 -05:00
Peter Somogyi 42ca976803 HBASE-20689 Docker fails to install rubocop for precommit
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-06 23:51:47 +02:00
Sean Busbey 9431ee3f49 HBASE-20501 update minimum supported Hadoop version to 2.7.1.
Signed-off-by: Andrew Purtell <apurtell@apache.org>

 Conflicts:
	src/main/asciidoc/_chapters/configuration.adoc
2018-05-30 21:28:06 -05:00
Sean Busbey 8a3b44a7e4 HBASE-20638 nightly source artifact testing should fail the stage if it's going to report an error on jira
Signed-off-by: Michael Stack <stack@apache.org>
2018-05-24 13:13:35 -05:00
Sean Busbey 29f492b344 HBASE-20591 nightly job needs to enable maven build system.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-05-22 17:43:21 -05:00
Nihal Jain 8c7293c514 HBASE-20438 Add an HBase antipattern check for reintroducing commons-logging
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 11:58:39 -05:00
Sean Busbey 53eb0f60c5 HBASE-20364 ensure jira comment from nightly reflects results of run comment comes from.
* in a post-step, build status can either be "null" or "SUCCESS" to indicate success.
* before we do an scm checkout for stages that post to the comment, set a default "we failed ¯\_(ツ)_/¯" comment.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-14 21:31:36 -05:00
Sean Busbey 1d133c005c HBASE-20389 Move website building flags into a profile.
Signed-off-by: Mike Drob <mdrob@apache.org>

 Conflicts:
	hbase-spark/pom.xml
	pom.xml
2018-04-13 22:50:55 -05:00
Sean Busbey dcd20e9cef HBASE-20335 ensure each stage of the nightly job gathers machine information.
* fix archiving for src tarball stage's machine info
* stop nightly wrapper desroying the output dir.

Signed-off-by: Michael Stack <stack@apache.org>
2018-04-13 22:45:19 -05:00
Sean Busbey bd2dddae60 HBASE-20379 shadedjars yetus plugin should add a footer link
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-13 10:48:05 -05:00
Sean Busbey d5d5c78849 HBASE-20112 register nightly junit over hadoop3 results with jenkins.
Signed-off-by: Michael Stack <stack@apache.org>
2018-04-13 10:48:05 -05:00
Mike Drob a50d9f4351 HBASE-20356 Make skipping protoc possible 2018-04-12 14:54:03 -05:00
Michael Stack cc801f5f2d HBASE-20385 Purge md5-making from our little make_rc.sh script 2018-04-11 09:18:43 -07:00
Sean Busbey 9bf087d28b HBASE-20335 nightly job bash cleanup.
* Ensure Jenkins steps that invoke bash inline set -e
* machine stats script should check that passed directory will work

Signed-off-by: Michael Stack <stack@apache.org>
2018-04-11 01:30:49 -05:00
Sean Busbey 1ef2450ca9 HBASE-20068 personality tests for Apache Yetus should use the maven plugin to exec maven.
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Reid Chan <reidddchan@outlook.com>
2018-04-11 01:30:46 -05:00
Sean Busbey 469b57bba0 HBASE-15466 test personality should recognize docs-only patch.
* add a new test to build the refguide specifically instead of site
* check for changes to src/main/asciidoc or src/main/xslt and run that test and only that test
* check for changes to the hbase-default.xml file and build the refguide if found (but maybe other tests too)
* fallback to relying on the yetus default for other changes
* fix some missing start_clock entries that cause longer-than-actual reported test time.

Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-09 10:37:49 -05:00
Apekshit Sharma 1e0d909532 HBASE-17730 Add license header to design doc. 2018-04-04 13:11:53 -07:00
Apekshit Sharma 40fbecd97c HBASE-17730 Migration to 2.0 for coprocessors. Upload AsciiDoc for coprocessor design improvements made in HBASE-17732. 2018-04-03 15:50:09 -07:00
Sean Busbey dba480856b HBASE-20308 Dockerfile for test must include git.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-28 18:39:06 -05:00
Mike Drob d42886f4fe HBASE-20153 Add error prone to precommit 2018-03-12 16:06:08 -05:00
Sean Busbey b87165cbf7 HBASE-20162 [nightly] depending on pipeline execution we sometimes refer to the wrong workspace
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-09 17:23:13 -06:00
Mike Drob 32c5118f6f HBASE-20164 add hadoopcheck link to table footer 2018-03-09 15:41:12 -06:00
Sean Busbey c99e6f7439 HBASE-18467 addendum parallel steps must account for SCM and marshalling results
* do a scm checkout on the stages that need access to source.
* ensure our install job runs on the ubuntu label
* copy jira comments to main workspace
* simplify the jira comment

Signed-off-by: Michael Stack <stack@apache.org>
2018-03-08 11:44:31 -06:00
Sean Busbey d4c4af09a1 HBASE-20075 remove logic for branch-1.1 nightly testing
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-08 11:44:31 -06:00
Sean Busbey 27642614b4 HBASE-15151 ensure findbugs check runs in nightly tests on all branches.
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-08 11:44:31 -06:00
Sean Busbey 44f1129234 HBASE-18467 report nightly results to devs via jira
- rely on parallel pipeline to ensure all stages always run
- define non-CPS jira commenting function
- comment on jiras in the changeset with summary and links

Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-08 11:44:31 -06:00
Sean Busbey f5c8713bdd HBASE-20134 dev-support scripts should use mktemp instead of hard-coding /tmp.
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-07 08:39:19 -06:00
Sean Busbey b59c39d942 HBASE-20070 addendum for missing echo. Website generation is failing. 2018-03-04 09:04:06 +05:30
Michael Stack bc735474a9 HBASE-20122 Up timeout on nightlies to 9 hours from 8 on branch-2 and
master
2018-03-03 07:33:25 -08:00
Sean Busbey 9927c2e14a HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-spark-it/pom.xml
2018-03-02 09:51:43 -06:00
Alex Leblang 2f3632a2a4 HBASE-20106 [api compliance chacker] Fix Bug Where Branch Isn't Found
While git rev-parse, sometimes the branch cannot be found unless
the remote is specified. This fix tries to use "origin" if the
remote is not specified and the branch is not found.

Signed-off-by: Sean Busbey <busbey@apache.org>
2018-02-28 11:54:31 -06:00
Josh Elser 7d5744d2f8 HBASE-20089 Use the ASF recommended naming for SHA512 xsum files
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-27 09:57:28 -05:00
Michael Stack b2fe241089 HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies; AMENDMENT hardcode proclimit and docker memlimit in nightly script... 2018-02-01 20:54:38 -08:00
Michael Stack 1dc640c296 HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies; REAPPLY TO TEST 2018-02-01 20:27:05 -08:00
Michael Stack 12a6af0de5 Revert "HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies"
This reverts commit 2e6bc12441.
2018-02-01 18:04:59 -08:00
Michael Stack ce6c3d96ea HBASE-19901 Up yetus proclimit on nightlies; ADDENDUM -- remove docker mem sizing for the moment... 2018-02-01 16:39:05 -08:00
Michael Stack fe6c06ad6c HBASE-19901 Up yetus proclimit on nightlies; ADDENDUM -- up proclimit from 6k to 10k and set Docker memory limit of 20G (default is 4G) 2018-02-01 14:08:24 -08:00
Michael Stack afdfa54cd3 HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies 2018-01-30 22:26:58 -08:00
Michael Stack 2835d51f9a Revert "HBASE-19899 Dump ulimit -a, fd count, and free output at end of build into system dir"
Reverting because doesn't work.

This reverts commit 44019c21fb.
2018-01-30 12:04:37 -08:00
Michael Stack 44019c21fb HBASE-19899 Dump ulimit -a, fd count, and free output at end of build into system dir 2018-01-30 11:54:47 -08:00
Michael Stack 8a1fca09cb HBASE-19892 Checking patch attach and yetus 0.7.0 and move to Yetus 0.7.0
One-liner that ups our yetus version from 0.6.0 to 0.7.0.
2018-01-29 22:35:47 -08:00
Michael Stack 03a418ed39 HBASE-19891 Up nightly test run timeout from 6 hours to 8 2018-01-29 22:15:37 -08:00
Apekshit Sharma 130da9d18b HBASE-19860 Run branch-2 nightly job more frequently to help with flaky tests debugging 2018-01-24 20:00:30 -08:00
Apekshit Sharma e60d302b90 HBASE-19789 Not exclude flaky tests from nightly builds. 2018-01-16 10:09:50 -08:00
Apekshit Sharma f91589d305 HBASE-19789 Exclude flaky tests from nightly branch-2 runs 2018-01-12 16:24:45 -08:00
Michael Stack 026f535a77 HBASE-19787 Fix or disable tests broken in branch-2 so can cut beta-1
M dev-support/make_rc.sh
  Disable checkstyle building site. Its an issue being fixed over in HBASE-19780

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  The clusterid was being set into the process only after the
  regionserver registers with the Master. That can be too late for some
  test clients in particular. e.g. TestZKAsyncRegistry needs it as soon
  as it goes to run which could be before Master had called its run
  method  which is regionserver run method which then calls back to the
  master to register itself... and only then do we set the clusterid.
  HBASE-19694 changed start order which made it so this test failed.
  Setting the clusterid right after we set it in zk makes the test pass.

  Another change was that backup masters were not going down on stop.
  Backup masters were sleeping for the default zk period which is 90
  seconds. They were not being woken up to check for stop. On stop
  master now tells active master manager.

M hbase-server/src/test/java/org/apache/hadoop/hbase/TestJMXConnectorServer.java
  Prevent creation of acl table. Messes up our being able to go down
  promptly.

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReadRequestMetrics.java
  Disabled for now because it wants to run with regions on the Master...
  currently broke!

M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestZKAsyncRegistry.java
  Add a bit of debugging.

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSAsyncFSWAL.java
  Disabled. Fails 40% of the time.

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSFSHLog.java
  Disabled. Fails 33% of the time.

Disabled stochastic load balancer for favored nodes because it fails on
occasion and we are not doing favored nodes in branch-2.
2018-01-12 14:09:23 -08:00
Michael Stack 581f168f6e
HBASE-19659 Enable -x in make_rc.sh so logs where it is in execution 2017-12-28 14:31:28 -08:00
Apekshit Sharma b263eb5648 HBASE-19570 Add hadoop3 tests to Nightly master/branch-2 runs.
Jenkins fails the whole build immediately if any stage fails. Hadoop2 tests run before Hadoop3 tests.
So Hadoop3 tests will run only if hadoop2 tests pass.
2017-12-21 17:44:33 -08:00