62 Commits

Author SHA1 Message Date
stack
450503989f HBASE-23779 Up the default fork count to make builds complete faster; make count relative to CPU count
ADDENDUM: Purge changes made to hbase-personality as they disstabilize
jenkins build. All else remains, the changing of forkcount for tests.
2020-02-07 16:00:50 -08:00
Nick Dimiduk
5cda68899a HBASE-23793 Increase maven heap allocation to 4G in Yetus personality (#1122)
I saw this over on
https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2/2447/console. Looks
like we need to bump the memory allocation for maven. I wonder if this
is the underlying cause of HBASE-22470.

```
6:38:47  ============================================================================
16:38:47  ============================================================================
16:38:47                                Finished build.
16:38:47  ============================================================================
16:38:47  ============================================================================
16:38:47
16:38:47
Post stage
[Pipeline] stash
16:38:48  Warning: overwriting stash 'hadoop2-result'
16:38:48  Stashed 1 file(s)
[Pipeline] junit
16:38:48  Recording test results
16:38:54  Remote call on H2 failed
Error when executing always post condition:
java.io.IOException: Remote call on H2 failed
	at hudson.remoting.Channel.call(Channel.java:963)
	at hudson.FilePath.act(FilePath.java:1072)
	at hudson.FilePath.act(FilePath.java:1061)
	at hudson.tasks.junit.JUnitParser.parseResult(JUnitParser.java:114)
	at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:137)
	at hudson.tasks.junit.JUnitResultArchiver.parseAndAttach(JUnitResultArchiver.java:167)
	at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:52)
	at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:25)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(XMLStringBuffer.java:208)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanData(XMLEntityScanner.java:1515)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanCDATASection(XMLDocumentFragmentScannerImpl.java:1654)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3014)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
	at org.dom4j.io.SAXReader.read(SAXReader.java:465)
	at org.dom4j.io.SAXReader.read(SAXReader.java:343)
	at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:178)
	at hudson.tasks.junit.TestResult.parse(TestResult.java:348)
	at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:281)
	at hudson.tasks.junit.TestResult.parse(TestResult.java:206)
	at hudson.tasks.junit.TestResult.parse(TestResult.java:178)
	at hudson.tasks.junit.TestResult.<init>(TestResult.java:143)
	at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:146)
	at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:118)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3052)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	... 4 more

[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
16:38:54  Failed in branch yetus jdk8 hadoop2 checks
```

Signed-off-by: stack <stack@apache.org>
2020-02-05 14:20:34 -08:00
Michael Stack
4c8ac72194
HBASE-23779 Up the default fork count; make count relative to CPU count (#1108)
Set the fork count for first and second parts to be 0.5C. Add a bit of
doc too on this as well as some qualification on our test categories.
Also adds -T0.5C to MAVEN_ARGS in the hbase personality.
2020-02-04 20:47:02 -08:00
Sean Busbey
82037ee3f2 HBASE-23228 Allow for jdk8 specific modules on branch-1 in precommit/nightly testing (#804)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-11-08 10:08:56 -06:00
Sean Busbey
5cedb09b65 HBASE-22955 Properly set jdk7 as JAVA_HOME for branches-1 precommit and nightly. (#621)
master/branches-2 specific changes: work around yetus overwriting JAVA_HOME
in the container with the host JAVA_HOME.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 41990ba20aab61379d6cf575b7eb2776eb55d883)
(cherry picked from commit bcad0d9f98b61141e82969686df93fcda171c61c)
2019-09-16 22:57:16 -05:00
Peter Somogyi
af86b7bd06 HBASE-22981 Remove unused flags for Yetus (#591)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-06 14:58:21 +02:00
Duo Zhang
48f0b1805e HBASE-22724 Add a emoji on the vote table for pre commit result on github (#568)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Stack <stack@apache.org>
2019-09-04 09:04:59 +08:00
Sean Busbey
72caf79bb3 HBASE-22518 yetus personality is treating branch-1.4 like earlier branches for hadoopcheck
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b4d5a9a8e11720b27fd556d6c28bcfdbcad7d20d)
2019-06-03 09:26:46 -05:00
Duo Zhang
59973d2024 HBASE-22500 Modify pom and jenkins jobs for hadoop versions
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-05-30 22:31:47 +08:00
Guanghao Zhang
c00cc4d9d6 HBASE-22462 (addendum) Should run a 'mvn install' at the end of hadoop check in pre commit job 2019-05-24 12:07:15 +08:00
Guanghao Zhang
b469d06bd4 HBASE-22462 Should run a 'mvn install' at the end of hadoop check in pre commit job 2019-05-24 12:07:09 +08:00
zhangduo
6a28e76f7e HBASE-22442 Nightly build is failing with hadoop 3.x 2019-05-19 07:56:02 +08:00
Duo Zhang
5537de38b3 HBASE-20970 Update hadoop check versions for hadoop3 in hbase-personality
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-05-17 09:51:59 +08:00
Duo Zhang
6bd6a953a7 HBASE-22399 Change default hadoop-two.version to 2.8.x and remove the 2.7.x hadoop checks 2019-05-14 16:45:32 +08:00
zhangduo
4562bf608c HBASE-22174 Remove error prone from our precommit javac check 2019-04-06 19:12:51 +08:00
zhangduo
a33b3401cf HBASE-22100 False positive for error prone warnings in pre commit job
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-03-28 13:06:53 +08:00
Peter Somogyi
0b086087b2 HBASE-21547 Precommit uses master flaky list for other branches
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-01-02 08:31:06 +01:00
Sean Busbey
f9a3d8acc4 HBASE-21058 When testing branches one, the PDF ref guide is named 'book.pdf'
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2018-08-21 09:46:03 -05:00
Sean Busbey
98da781ecf HBASE-21074 JDK7 builds need to be done with TLSv1.2.
* for master and branches-2 we skip the jdk7 stuff

Signed-off-by: Michael Stack <stack@apache.org>
2018-08-21 09:15:26 -05:00
Yu Li
492c0fcd2e HBASE-20838 Include hbase-server in precommit test if CommonFSUtils is changed 2018-07-12 10:17:38 +08:00
Mike Drob
b04c976fe6 HBASE-20478 Update checkstyle to v8.2
Cannot go to latest (8.9) yet due to
  https://github.com/checkstyle/checkstyle/issues/5279

* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors

Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-18 14:02:40 -07:00
Sean Busbey
a4cb98b1cd HBASE-20733 QABot should run checkstyle tests if the checkstyle configs change
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-14 20:26:42 -05:00
Sean Busbey
9431ee3f49 HBASE-20501 update minimum supported Hadoop version to 2.7.1.
Signed-off-by: Andrew Purtell <apurtell@apache.org>

 Conflicts:
	src/main/asciidoc/_chapters/configuration.adoc
2018-05-30 21:28:06 -05:00
Nihal Jain
8c7293c514 HBASE-20438 Add an HBase antipattern check for reintroducing commons-logging
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 11:58:39 -05:00
Sean Busbey
bd2dddae60 HBASE-20379 shadedjars yetus plugin should add a footer link
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-13 10:48:05 -05:00
Mike Drob
a50d9f4351 HBASE-20356 Make skipping protoc possible 2018-04-12 14:54:03 -05:00
Sean Busbey
1ef2450ca9 HBASE-20068 personality tests for Apache Yetus should use the maven plugin to exec maven.
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Reid Chan <reidddchan@outlook.com>
2018-04-11 01:30:46 -05:00
Sean Busbey
469b57bba0 HBASE-15466 test personality should recognize docs-only patch.
* add a new test to build the refguide specifically instead of site
* check for changes to src/main/asciidoc or src/main/xslt and run that test and only that test
* check for changes to the hbase-default.xml file and build the refguide if found (but maybe other tests too)
* fallback to relying on the yetus default for other changes
* fix some missing start_clock entries that cause longer-than-actual reported test time.

Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-09 10:37:49 -05:00
Mike Drob
d42886f4fe HBASE-20153 Add error prone to precommit 2018-03-12 16:06:08 -05:00
Mike Drob
32c5118f6f HBASE-20164 add hadoopcheck link to table footer 2018-03-09 15:41:12 -06:00
Michael Stack
1dc640c296 HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies; REAPPLY TO TEST 2018-02-01 20:27:05 -08:00
Michael Stack
12a6af0de5 Revert "HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies"
This reverts commit 2e6bc12441f50a4de9ee87e96132cb067556a528.
2018-02-01 18:04:59 -08:00
Michael Stack
ce6c3d96ea HBASE-19901 Up yetus proclimit on nightlies; ADDENDUM -- remove docker mem sizing for the moment... 2018-02-01 16:39:05 -08:00
Michael Stack
fe6c06ad6c HBASE-19901 Up yetus proclimit on nightlies; ADDENDUM -- up proclimit from 6k to 10k and set Docker memory limit of 20G (default is 4G) 2018-02-01 14:08:24 -08:00
Michael Stack
afdfa54cd3 HBASE-19901 HBASE-19901 Up yetus proclimit on nightlies 2018-01-30 22:26:58 -08:00
Apekshit Sharma
b263eb5648 HBASE-19570 Add hadoop3 tests to Nightly master/branch-2 runs.
Jenkins fails the whole build immediately if any stage fails. Hadoop2 tests run before Hadoop3 tests.
So Hadoop3 tests will run only if hadoop2 tests pass.
2017-12-21 17:44:33 -08:00
Apekshit Sharma
90ac3c93c9 HBASE-19491 Improvements to Nighly runs: Fixes findbugs tests, Exclude flaky tests from master. 2017-12-20 17:59:27 -08:00
Apekshit Sharma
6616a3fd2d HBASE-19526 Update hadoop version to 3.0 GA 2017-12-17 22:52:54 -08:00
Apekshit Sharma
9144452773 HBASE-19489 Adds QUICK_HADOOPCHECK flag to check against only the latest maintenance releases in "hadoopcheck". Updates hadoop version to 3.0.0-beta1. 2017-12-13 00:36:34 -08:00
Sean Busbey
d5df4c0426 HBASE-19249 test for "hbase antipatterns" should check _count_ of occurance rather than text of
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-11-13 23:05:25 -06:00
Andrew Purtell
e3bd475317 HBASE-19203 Update Hadoop version used for build to 2.7.4 (from 2.5.1) 2017-11-07 12:37:53 -08:00
Mike Drob
34df2e665e HBASE-16338 Remove Jackson1 deps
* Change imports from org.codehaus to com.fasterxml
* Exclude transitive jackson1 from hadoop and others
* Minor test cleanup to add assert messages, fix some parameter order
* Add anti-pattern check for using jackson 1 imports
* Add explicit non-null serialization directive to ScannerModel
2017-10-23 15:24:51 -05:00
Sean Busbey
a37ff39b7a HBASE-19039 refactor shadedjars test to only run on java changes.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-20 19:43:08 -05:00
Sean Busbey
01ab747a5f HBASE-19060 precommit plugin test 'hadoopcheck' should only run when java or maven files change.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-20 19:43:02 -05:00
Mike Drob
723eb7c6ba HBASE-19038 precommit mvn install should run from root on patch 2017-10-18 10:39:52 -05:00
Mike Drob
822cc1ed73 HBASE-18505 Yetus only run root when all modules changed 2017-10-13 12:12:38 -05:00
Sean Busbey
e4647da0db HBASE-18934 pick hadoop versions for precommit test after patch branch is determined.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-09 11:38:16 -05:00
Josh Elser
8c6ed571bd HBASE-17441 Fix invalid quoting around hadoop-3 build in yetus personality 2017-10-01 15:21:54 -04:00
Sean Busbey
ea610cfcf7 HBASE-18833 Ensure precommit personality is up to date on all active branches
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-09-20 05:51:42 -05:00
Sean Busbey
5ed2f899e1 HBASE-18834 fix shellcheck warning on hbase personality
Signed-off-by: Dima Spivak <dimaspivak@apache.org>
2017-09-18 08:55:30 -05:00