Commit Graph

16962 Commits

Author SHA1 Message Date
stack 9067dfe83c HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Overwrite yarn configs that write /tmp dir by default.
2020-04-11 21:23:17 -07:00
stack cc4aaefa52 HBASE-24167 [Flakey Tests] TestHRegionWithInMemoryFlush#testWritesWhileScanning gets stuck MVCC 2020-04-10 18:30:08 -07:00
stack 3ccff5784e HBASE-24162 Move CHANGES.txt to CHANGES.md. Add RELEASENOTES.md too on branch-2, branch-2.3, and master 2020-04-10 14:41:02 -07:00
huaxiangsun 3e4dfa9b31
HBASE-24161 [flakey test] locking.TestEntityLocks.testEntityLockTimeout (#1477) (#1480)
Signed-off-by: stack <stack@apache.org>
2020-04-10 09:42:47 -07:00
huaxiangsun 39187f87dd
HBASE-24159 [flakey test] regionserver.TestRegionMergeTransactionOnCluster (#1472) (#1474)
Delete tables after each test cases to avoid polluting states.

Signed-off-by: stack <stack@apache.org>
2020-04-09 21:09:34 -07:00
Pankaj 823cf97495 HBASE-24074 ConcurrentModificationException occured in ReplicationSourceManager while refreshing the peer (#1471)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-09 16:42:22 -07:00
Bharath Vissapragada 91aa3ce97c HBASE-24156: Switch the default registry to ZK based
In branch-2 we want to retain the ZK based registry as the default.
2020-04-09 16:13:45 -07:00
Bharath Vissapragada db7ad07081 Revert "Switch the default registry to ZK based"
This reverts commit 558ee079fd because
it wasn't tagged with a jira ID. Will be re-applied.
2020-04-09 16:13:45 -07:00
Michael Stack 9eb49172d0
HBASE-24150 Allow module tests run in parallel (#1464)
Pass --threads=2 to mvn when yetus runs so some parallelism
when dependencies allow.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-09 14:41:41 -07:00
stack 2d11796380 HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded 2020-04-09 11:03:22 -07:00
Reid Chan 5012241817 HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be (#1439)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-04-09 16:30:00 +08:00
Sean Busbey 611c62f52a HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer (#1455)
- at DEBUG log messages about RegionCountSkewCostFunction region/server totals
- at DEBUG log messages about the decision to balance or not with total costs
- at TRACE log messages about region count on each server RegionCountSkewCostFunction sees
- at TRACE log message with the individual cost functions used in the decision to balance or not

Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 2d78a286b6)
2020-04-08 15:50:38 -05:00
huaxiangsun 6f00714ad3
HBASE-24120 Flakey Test: TestReplicationAdminWithClusters timeout (#1441) (#1456)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-08 12:23:50 -07:00
Andrew Purtell 3f511b0e93
HBASE-24099 Use a fair ReentrantReadWriteLock for the region close lock (#1407)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-04-08 11:40:42 -07:00
Michael Stack 5db62b422c
HBASE-24131 [Flakey Tests] TestExportSnapshot takes too long; up against 13min max (#1452)
Split TestExportSnapshot and TestVerifyReplication to two smaller tests
rather than one big one that can take 13minutes+ when contention.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-04-08 10:57:18 -07:00
Michael Stack ed54c35cf0
HBASE-24134 Down forked JVM heap size from 2800m to 2200m for jdk8 and jdk11 (#1451)
Down jdk8 forked jvm heap from 2800 to 2200 and the jdk11 heap from
3200 to 2200. Down the mvn size from 4G to 3.6G

Change how many puts done by TestMultiRespectsLimits because made
the test run the forked heap over 2.5G in size.

Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-08 10:51:03 -07:00
stack 953c326498 HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Addendum number 3; set the yetus proclimit.
2020-04-07 10:22:24 -07:00
stack ba87ffed09 Revert "HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread."
Revert because applied with wrong JIRA number.

This reverts commit 9c60c5fddb.
2020-04-07 10:17:48 -07:00
Michael Stack 9da7f95fa7
HBASE-24128 [Flakey Tests] Add retry on thrift cmdline if client fails plus misc debug (#1442)
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAssignment.java
 Saw case where Master failed startup but it came out as an IOE so we
 did not trip the retry logic.

hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
 Add some debug and up timeouts. This test fails frequently for me
 locally.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/locking/TestEntityLocks.java
 Up the wait from 2x 200ms to 10x in case a pause on hardware or GC.
 This test fails locally and up on jenkins.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClearRegionBlockCache.java
 Debug. Have assert say what bad count was.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 Fails on occasion. Found count is off by a few. Tricky to debug. HBASE-24129 to reenable.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
 Debug. Add wait and check before moving to assert.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Check for null before shutting; can be null if failed start.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
 Add retry if client messes up connection. Fails for me locally.
2020-04-07 09:28:05 -07:00
Minwoo Kang af659564f3 HBASE-24130 rat plugin complains about having an unlicensed file (#1445)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-07 09:18:27 -07:00
Duo Zhang 74a85e26ee HBASE-24055 Make AsyncFSWAL can run on EC cluster (#1437)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-07 23:46:06 +08:00
stack 9c60c5fddb HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread.
Up the nproc in hbase-personality. Add test that prints out env.
2020-04-07 07:37:48 -07:00
binlijin 4a85f06590 HBASE-24077 When encounter RowTooBigException, log the row info. (#1379)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-07 10:36:40 +08:00
stack 01715fbbf6 HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Addendum number 2; docker doesn't like the ulimit arg as is.
2020-04-06 18:24:09 -07:00
Andrew Purtell 2905507ce2
HBASE-24115 Relocate test-only REST "client" from src/ to test/ and mark Private (#1434)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-06 17:06:04 -07:00
Nick Dimiduk 099333079b HBASE-24081 Provide documentation for running Yetus with HBase
Signed-off-by: stack <stack@apache.org>
2020-04-06 16:40:24 -07:00
stack 25773c681b HBASE-24126 Up the container nproc uplimit from 10000 to 12500
ADDENDUM: Add to hbase-personality
2020-04-06 16:29:46 -07:00
stack ed24998b85 HBASE-24126 Up the container nproc uplimit from 10000 to 12500 2020-04-06 13:44:53 -07:00
stack 389dfd2198 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
Reapply but as an @Ignore for the flakey test.
2020-04-06 12:40:25 -07:00
stack 82773a8c6e Revert "HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash"
Reverting in favor of adding an @Ignore on this test until
root cause of flakyness HBASE-24117 is addressed.

This reverts commit 9985c06647.
2020-04-06 12:40:13 -07:00
stack 4d31af23e8 HBASE-24122 Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory' 2020-04-06 08:58:16 -07:00
Peter Somogyi b7d67dcc38 HBASE-24111 Enable CompactionTool executions on non-HDFS filesystems (#1427)
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-06 11:55:35 +02:00
huaxiangsun 9d28f2d086 HBASE-24105 [Flakey Test] regionserver.TestRegionReplicas (#1425)
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-05 13:23:33 -07:00
Michael Stack 3030b59e29
HBASE-24113 Upgrade the maven we use from 3.5.4 to 3.6.3 in nightlies (#1430)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-05 12:53:44 -07:00
stack 9845f9e416 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash 2020-04-04 17:45:22 -07:00
stack 51485db67c HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController
Addendum, make it three seconds.
2020-04-04 17:38:54 -07:00
Huaxiang Sun 93c3653ecf HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController 2020-04-04 13:14:30 -07:00
Nick Dimiduk 1df979df18 HBASE-24049 use hadoop-2.10.0 for "packaging and integration" check
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-03 16:59:54 -07:00
Nick Dimiduk 4e31ada43b HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-03 09:24:29 -07:00
Viraj Jasani 73aded09ec
HBASE-24102 : Remove decommissioned RS from target servers while unlo… (#1417)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-03 18:36:02 +05:30
stack 7f353cc40d HBASE-24107 [Flakey Test] TestThriftServerCmdLine.testRunThriftServer NPEs if InfoServer port clash 2020-04-02 20:29:58 -07:00
huaxiangsun 53299a6de2
HBASE-24080 [flakey test] TestRegionReplicaFailover.testSecondaryRegionKill fails. (#1421) (#1423)
Signed-off-by: stack <stack@apache.org>
2020-04-02 18:21:40 -07:00
meiyi c97c9e2eda HBASE-24103 [Flakey Tests] TestSnapshotScannerHDFSAclController (#1416)
Signed-off-by: stack <stack@apache.org>
2020-04-02 09:56:26 -07:00
Nick Dimiduk 3be20c69a8 HBASE-24084 Fix missing jdk8 dependencies in hbase-assembly/hadoop-two-compat
After HBASE-23986, running on JDK8/Hadoop2 from the binary tarball
results in

```
Exception in thread "main" java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3062)
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 2 more
```

The missing class is from `com.fasterxml.woodstox:woodstox-core`. This
and `org.codehaus.woodstox:stax2-api` are explicit dependencies of
`org.apache.hadoop:hadoop-common:jar:2.10.0`. Thus, include them in
the default jars provided under `lib`.

Signed-off-by: stack <stack@apache.org>
2020-04-02 09:53:29 -07:00
niuyulin 75714a4a06 HBASE-24021 Fail fast when bulkLoadHFiles method catch some IOException (#1343)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-04-02 23:26:51 +08:00
huaxiangsun 231c2bca94 HBASE-24073 [flakey test] client.TestAsyncRegionAdminApi messed up compaction state. (#1414)
Addendum:
  For major compaction test, set hbase.hstore.compaction.min to a big number to
  avoid kicking in minor compactions, which will pollute compaction state and
  sometimes, cause major compaction cannot happen.

Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-02 08:18:17 -07:00
stack e6264c9573 HBASE-24100 [Flakey Tests] Add test to check we work properly when port clash setting up thriftserver
Addendum; add missing @ClassRule
2020-04-01 17:49:35 -07:00
stack 97a47ed0ee HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread
Set the forkcount down from 0.5C to 0.25C.
2020-04-01 16:05:59 -07:00
stack 09141681f6 Revert "HBASE-24051 Allows indirect inheritance to CanUnbuffer (#1406)"
This reverts commit 30f5852fc2.
2020-04-01 15:49:11 -07:00
申胜利 30f5852fc2 HBASE-24051 Allows indirect inheritance to CanUnbuffer (#1406)
Signed-off-by: stack <stack@apache.org>
2020-04-01 14:41:14 -07:00