16855 Commits

Author SHA1 Message Date
Michael Stack
c5de732006 HBASE-24126 Up the container nproc uplimit from 10000 to 12500 (#1504) 2020-04-13 15:00:33 -07:00
Jianfei Jiang
2e1a719e26 HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7 (#1453)
* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

Co-authored-by: 姜建飞 10222269 <jiang.jianfei@zte.com.cn>

Signed-off-by: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Signed-off-by: Norbert Kalmar <nkalmar@cloudera.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangdo@apache.org>
2020-04-13 13:49:12 -07:00
Michael Stack
f2f86c1790 HBASE-24134 Down forked JVM heap size from 2800m to 2200m for jdk8 and jdk11 (#1451)
Down jdk8 forked jvm heap from 2800 to 2200 and the jdk11 heap from
3200 to 2200. Down the mvn size from 4G to 3.6G

Change how many puts done by TestMultiRespectsLimits because made
the test run the forked heap over 2.5G in size.

Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-13 12:05:20 -07:00
Michael Stack
49d5bd3c39 HBASE-24150 Allow module tests run in parallel (#1464)
Pass --threads=2 to mvn when yetus runs so some parallelism
when dependencies allow.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-13 11:54:50 -07:00
stack
7b2b6dcf97 HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded 2020-04-13 11:52:12 -07:00
stack
117164e12f HBASE-24167 [Flakey Tests] TestHRegionWithInMemoryFlush#testWritesWhileScanning gets stuck MVCC 2020-04-13 11:44:44 -07:00
stack
0931019681 Revert "HBASE-24126 Up the container nproc uplimit from 10000 to 12500 (#1450)"
This reverts commit f5b3945ea4e0fdd5f8fa521db34fd39fc74ec3ba.
2020-04-13 11:44:34 -07:00
stack
f6ab525661 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Overwrite yarn configs that write /tmp dir by default.
2020-04-13 11:30:39 -07:00
Joseph295
80011db571
HBASE-24168 Align the internal.protobuf.version in hbase-protocol-shaded with hbase-thirdparty (#1488)
Co-authored-by: Joseph295 <johnhomsea@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-13 08:50:18 +08:00
Viraj Jasani
cd259d7364
HBASE-24140 : Move CandidateGenerator and their implementors out of StochasticLoadBalancer (#1458)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-12 19:26:18 +05:30
Jan Hentschel
5dfd0d8e04
HBASE-23998. Update license for jetty-client. (#1297) (#1498)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-04-12 12:05:03 +02:00
Michael Stack
f5b3945ea4 HBASE-24126 Up the container nproc uplimit from 10000 to 12500 (#1450)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-10 15:58:38 -07:00
stack
a404ae128e HBASE-24162 Move CHANGES.txt to CHANGES.md. Add RELEASENOTES.md too on branch-2, branch-2.3, and master 2020-04-10 14:42:06 -07:00
huaxiangsun
5e11ff100a
HBASE-24161 [flakey test] locking.TestEntityLocks.testEntityLockTimeout (#1477) (#1481)
Signed-off-by: stack <stack@apache.org>
2020-04-10 09:40:50 -07:00
huaxiangsun
eae7523cef
HBASE-24159 [flakey test] regionserver.TestRegionMergeTransactionOnCluster (#1472) (#1475)
Delete tables after each test cases to avoid polluting states.

Signed-off-by: stack <stack@apache.org>
2020-04-09 21:08:10 -07:00
Pankaj
8fa93ef858 HBASE-24074 ConcurrentModificationException occured in ReplicationSourceManager while refreshing the peer (#1471)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-09 16:42:46 -07:00
Bharath Vissapragada
2365a4fb1b HBASE-24156: Switch the default registry to ZK based
In branch-2 we want to retain the ZK based registry as the default.
2020-04-09 16:14:11 -07:00
Bharath Vissapragada
b33c200a28 Revert "Switch the default registry to ZK based"
This reverts commit 558ee079fd04dfab8e61eca10ee98ab5bac89dfa because
it was not tagged with a jira ID. Will be re-applied.
2020-04-09 16:14:11 -07:00
Reid Chan
436567e34d HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be (#1439)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-04-09 16:49:48 +08:00
huaxiangsun
5b1f908027
HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (#1433) (#1459)
Signed-off-by: stack <stack@apache.org>
2020-04-08 14:11:08 -07:00
Sean Busbey
0822b6d167 HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer (#1455)
- at DEBUG log messages about RegionCountSkewCostFunction region/server totals
- at DEBUG log messages about the decision to balance or not with total costs
- at TRACE log messages about region count on each server RegionCountSkewCostFunction sees
- at TRACE log message with the individual cost functions used in the decision to balance or not

Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 2d78a286b682b34e8a718c521e265d58536f1e5e)
2020-04-08 15:49:29 -05:00
huaxiangsun
52103a11c9
HBASE-24120 Flakey Test: TestReplicationAdminWithClusters timeout (#1441) (#1457)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-08 12:24:42 -07:00
Andrew Purtell
b2ab435fce
HBASE-24099 Use a fair ReentrantReadWriteLock for the region close lock (#1407)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-04-08 11:40:27 -07:00
Michael Stack
45ac3b0dc5 HBASE-24131 [Flakey Tests] TestExportSnapshot takes too long; up against 13min max (#1452)
Split TestExportSnapshot and TestVerifyReplication to two smaller tests
rather than one big one that can take 13minutes+ when contention.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-04-08 11:02:59 -07:00
Michael Stack
0fbb08abe3 HBASE-24128 [Flakey Tests] Add retry on thrift cmdline if client fails plus misc debug (#1442)
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAssignment.java
 Saw case where Master failed startup but it came out as an IOE so we
 did not trip the retry logic.

hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
 Add some debug and up timeouts. This test fails frequently for me
 locally.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/locking/TestEntityLocks.java
 Up the wait from 2x 200ms to 10x in case a pause on hardware or GC.
 This test fails locally and up on jenkins.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClearRegionBlockCache.java
 Debug. Have assert say what bad count was.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 Fails on occasion. Found count is off by a few. Tricky to debug. HBASE-24129 to reenable.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
 Debug. Add wait and check before moving to assert.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Check for null before shutting; can be null if failed start.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
 Add retry if client messes up connection. Fails for me locally.
2020-04-07 09:28:42 -07:00
Duo Zhang
f97b411b7e HBASE-24055 Make AsyncFSWAL can run on EC cluster (#1437)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-07 23:46:21 +08:00
Andrew Purtell
86d2ce5539
HBASE-24115 Relocate test-only REST "client" from src/ to test/ and mark Private (#1434)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-06 16:20:38 -07:00
stack
71bcfea4c1 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
Reapply but as an @Ignore for the flakey test.
2020-04-06 12:41:21 -07:00
stack
b75eeb104d Revert "HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash"
Reverting in favor of adding an @Ignore on this test until
root cause of flakyness HBASE-24117 is addressed.

This reverts commit 9985c06647ef7169b9940cd3b6b24c2ac6a322fc.
2020-04-06 12:41:16 -07:00
stack
6f23e09bc4 HBASE-24122 Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory' 2020-04-06 10:14:46 -07:00
Peter Somogyi
b6177569a1 HBASE-24111 Enable CompactionTool executions on non-HDFS filesystems (#1427)
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-06 12:02:04 +02:00
huaxiangsun
5a67fb8c83 HBASE-24105 [Flakey Test] regionserver.TestRegionReplicas (#1425)
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-05 13:23:55 -07:00
Michael Stack
14a50b106e HBASE-24113 Upgrade the maven we use from 3.5.4 to 3.6.3 in nightlies (#1430)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-05 12:55:19 -07:00
stack
26ec491d7d HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash 2020-04-04 20:12:34 -07:00
Nick Dimiduk
4df2ff9e17 HBASE-24049 use hadoop-2.10.0 for "packaging and integration" check
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-03 17:01:50 -07:00
Nick Dimiduk
5de9d71200 HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-03 09:36:43 -07:00
Viraj Jasani
6cce123324
HBASE-24102 : Remove decommissioned RS from target servers while unlo… (#1417)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-03 18:40:42 +05:30
stack
7d3bf91104 HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread
Set the forkcount down from 0.5C to 0.25C.
2020-04-02 21:07:00 -07:00
stack
ce12466d91 HBASE-24107 [Flakey Test] TestThriftServerCmdLine.testRunThriftServer NPEs if InfoServer port clash 2020-04-02 20:30:33 -07:00
huaxiangsun
0893ac3c3e
HBASE-24080 [flakey test] TestRegionReplicaFailover.testSecondaryRegionKill fails. (#1421) (#1424)
Signed-off-by: stack <stack@apache.org>
2020-04-02 18:22:44 -07:00
meiyi
5f8dea5e9b HBASE-24103 [Flakey Tests] TestSnapshotScannerHDFSAclController (#1416)
Signed-off-by: stack <stack@apache.org>
2020-04-02 09:56:54 -07:00
Nick Dimiduk
2109415916 HBASE-24084 Fix missing jdk8 dependencies in hbase-assembly/hadoop-two-compat
After HBASE-23986, running on JDK8/Hadoop2 from the binary tarball
results in

```
Exception in thread "main" java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3062)
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 2 more
```

The missing class is from `com.fasterxml.woodstox:woodstox-core`. This
and `org.codehaus.woodstox:stax2-api` are explicit dependencies of
`org.apache.hadoop:hadoop-common:jar:2.10.0`. Thus, include them in
the default jars provided under `lib`.

Signed-off-by: stack <stack@apache.org>
2020-04-02 09:56:13 -07:00
niuyulin
4da6402298 HBASE-24021 Fail fast when bulkLoadHFiles method catch some IOException (#1343)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-04-02 23:29:25 +08:00
huaxiangsun
253692cccd HBASE-24073 [flakey test] client.TestAsyncRegionAdminApi messed up compaction state. (#1414)
Addendum:
  For major compaction test, set hbase.hstore.compaction.min to a big number to
  avoid kicking in minor compactions, which will pollute compaction state and
  sometimes, cause major compaction cannot happen.

Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-02 08:17:50 -07:00
stack
9e5a9bed25 HBASE-24100 [Flakey Tests] Add test to check we work properly when port clash setting up thriftserver
Addendum; add missing @ClassRule
2020-04-01 17:50:55 -07:00
stack
e6327398bd Revert "HBASE-24051 Allows indirect inheritance to CanUnbuffer (#1406)"
This reverts commit 34aca9457387023128b3d3879794b9cf0808eb7b.
2020-04-01 15:49:45 -07:00
申胜利
34aca94573 HBASE-24051 Allows indirect inheritance to CanUnbuffer (#1406)
Signed-off-by: stack <stack@apache.org>
2020-04-01 14:41:51 -07:00
Nick Dimiduk
75f9d8c2eb HBASE-24071 [JDK11] Remove unit filter from nightly and precommit jobs
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-01 12:31:55 -07:00
stack
0532ee8f2c HBASE-24100 [Flakey Tests] Add test to check we work properly when port clash setting up thriftserver 2020-04-01 12:24:36 -07:00
Nick Dimiduk
4391776279 HBASE-24092 Fix links to build reports generated by nightly job
Signed-off-by: stack <stack@apache.org>
2020-04-01 12:22:22 -07:00