Commit Graph

8127 Commits

Author SHA1 Message Date
niuyulin 6cea5c6462
HBASE-24222 remove FSUtils.checkAccess and replace with FileSystem.access in HBCK (#1557)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-23 23:20:38 +08:00
Viraj Jasani 4ed15e51d9
HBASE-24024 : Reject multi() requests with rows higher than threshold (#1560)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-22 12:46:40 +05:30
Jan Hentschel 75c717d4c2
HBASE-23848 Removed deprecated setStopRow from Scan (#1184)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-22 15:15:17 +08:00
niuyulin bc9184ee00
HBASE-23933 Separate a hbase-balancer module (#1436)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-22 14:55:38 +08:00
Beata Sudi 5f45c8293d
HBASE-24139 : Balancer should avoid leaving idle region servers (#1511)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-04-22 09:51:53 +05:30
stack e7b5c34d28 HBASE-24226 Address other hard references to '/tmp' found in Configuration 2020-04-21 17:40:35 -07:00
Viraj Jasani 5c53b606a9
HBASE-24175 : createDirAndSetProperty() (ADDENDUM) 2020-04-21 23:09:07 +05:30
stack 7c2f13eb70 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum #3; catch more '/tmp' references by hadoop.
2020-04-21 09:41:24 -07:00
stack 2d2e1d965d HBASE-24220 Allow that zk NOTEMPTY multi exception is retryable by running in-series
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java
 Cleanup checkstyle warnings. Don't depend on hbase-client
 ScannerCallable.

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
 Cut down on cluster resource usage.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/AbstractTestDLS.java
 Debug

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController3.java
 Debug

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
 Allow that NONEMPTY is retryable by running in series.
2020-04-20 15:51:11 -07:00
Guangxu Cheng bcacc4ce93
HBASE-23896 Snapshot owner cannot delete snapshot when ACL is enabled and Kerberos is not enabled (#1211)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-04-20 09:59:06 +08:00
XinSun 8e5c6da3f0
HBASE-24166 Duplicate implementation for acquireLock between CreateTableProcedure and its parent class (#1486)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-18 10:30:12 +08:00
Joseph295 5ff4376d80
HBASE-24182 log when the region is set to closing status (#1509)
Signed-off-by: stack <stack@apache.org>
2020-04-17 12:20:12 -07:00
Reid Chan db36cf71c5
HBASE-24174 Fix findbugs warning on ServiceAuthorizationManager for master branch (#1537)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-18 01:24:31 +08:00
gkanade 12aad76109
MetricsTable#updateFlushTime is wrong (#1539)
Authored-by: Gaurav Kanade <gkanade@microsoft.com>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-04-17 15:40:27 +05:30
Viraj Jasani d212dc4df0
HBASE-24195 : Admin.getRegionServers() should return live servers exc… (#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-16 20:13:14 +05:30
BukrosSzabolcs a0ef4cb8a2
HBASE-24186: RegionMover ignores replicationId (#1512)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-16 19:53:51 +05:30
Duo Zhang 1f66806c96
HBASE-24170 Remove hadoop-2.0 profile (#1495)
Signed-off-by: stack <stack@apache.org>
2020-04-16 18:57:40 +08:00
stack 853a72bc46 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum: add test to check for '/tmp' references.
2020-04-15 10:55:40 -07:00
huaxiangsun 3421423dd6
HBASE-24183 [flakey test] replication.TestAddToSerialReplicationPeer (#1514)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 10:07:57 -07:00
GeorryHuang 38954edb25 HBASE-23994: Add WebUI to Canary (#1292)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-14 15:56:19 -07:00
Nick Dimiduk cb60d23cb3 Revert "HBASE-23994:Add WebUI to Canary (#1292)"
This reverts commit daf79de329.
2020-04-14 15:55:56 -07:00
GeorryHuang daf79de329
HBASE-23994:Add WebUI to Canary (#1292)
HBASE-23994: Add WebUI to Canary

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-14 15:52:22 -07:00
binlijin 92b30f2638
HBASE-24181 Add region info when log meessages in HRegion. (#1506)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-14 19:36:16 +08:00
Michael Stack 5f08311b23
HBASE-24134 Down forked JVM heap size from 2800m to 2200m for jdk8 and jdk11 (#1451) (#1503)
Down jdk8 forked jvm heap from 2800 to 2200 and the jdk11 heap from
3200 to 2200. Down the mvn size from 4G to 3.6G

Change how many puts done by TestMultiRespectsLimits because made
the test run the forked heap over 2.5G in size.

Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-13 21:35:43 -07:00
Jianfei Jiang ba34a2ca30
HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7 (#1453)
* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

Co-authored-by: 姜建飞 10222269 <jiang.jianfei@zte.com.cn>

Signed-off-by: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Signed-off-by: Norbert Kalmar <nkalmar@cloudera.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangdo@apache.org>
2020-04-13 13:46:37 -07:00
stack 67b2e881ca HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded 2020-04-13 11:51:47 -07:00
stack 64c7fc6a52 HBASE-24167 [Flakey Tests] TestHRegionWithInMemoryFlush#testWritesWhileScanning gets stuck MVCC 2020-04-13 11:44:07 -07:00
stack b00b5270f6 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Overwrite yarn configs that write /tmp dir by default.
2020-04-13 11:29:25 -07:00
Reid Chan 22675d8070
HBASE-24112 [RSGroup] Support renaming rsgroup (#1435)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-13 14:04:32 +08:00
stack 72764cf628 HBASE-24153 Revert "Revert "Remove unnecessary super() in MultiVersionConcurrencyControl#MultiVersionConcurrencyControl() (#1468)""
A revert of a revert; i.e. a reapplication to add missing JIRA number

This reverts commit c4026d1beb.
2020-04-12 18:30:38 -07:00
stack c4026d1beb Revert "Remove unnecessary super() in MultiVersionConcurrencyControl#MultiVersionConcurrencyControl() (#1468)"
Forgot to add JIRA number. Reverting to add.

This reverts commit 12a0c1d5fe.
2020-04-12 18:30:11 -07:00
Viraj Jasani b6b8c6973d
HBASE-24140 : Move CandidateGenerator and their implementors out of StochasticLoadBalancer (#1458)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-12 19:13:47 +05:30
leosunli 12a0c1d5fe
Remove unnecessary super() in MultiVersionConcurrencyControl#MultiVersionConcurrencyControl() (#1468)
Signed-off-by: sunlisheng <lisheng.sun08@gmail.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-10 15:43:10 -07:00
Sean Busbey 09d20fb4d7
HBASE-24163 MOB compactor implementations should use format specifiers when calling String.format (#1479)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-10 11:00:53 -05:00
huaxiangsun ac4e94e868
HBASE-24161 [flakey test] locking.TestEntityLocks.testEntityLockTimeout (#1477)
Signed-off-by: stack <stack@apache.org>
2020-04-09 21:13:06 -07:00
Pankaj c92dd28a88
HBASE-24074 ConcurrentModificationException occured in ReplicationSourceManager while refreshing the peer (#1471)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-09 16:40:15 -07:00
huaxiangsun 8aa3304427
HBASE-24159 [flakey test] regionserver.TestRegionMergeTransactionOnCluster (#1472)
Delete tables after each test cases to avoid polluting states.

Signed-off-by: stack <stack@apache.org>
2020-04-09 16:27:19 -07:00
Reid Chan 1fb693b998
HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be (#1439)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-04-09 16:28:22 +08:00
Sean Busbey eb7df0498c
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446)
* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.

Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2020-04-08 14:16:43 -05:00
Sean Busbey 2d78a286b6
HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer (#1455)
- at DEBUG log messages about RegionCountSkewCostFunction region/server totals
- at DEBUG log messages about the decision to balance or not with total costs
- at TRACE log messages about region count on each server RegionCountSkewCostFunction sees
- at TRACE log message with the individual cost functions used in the decision to balance or not

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-08 14:03:33 -05:00
Andrew Purtell ac6060c3f9
HBASE-24099 Use a fair ReentrantReadWriteLock for the region close lock (#1407)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-04-08 11:36:48 -07:00
Wei-Chiu Chuang 72727ff9be
HBASE-23833. The relocated hadoop-thirdparty protobuf breaks HBase asyncwal (#1301)
* Use Reflection to access shaded Hadoop protobuf classes.

(cherry picked from commit a321e536989083ca3620bf2c53f12c07740bf5b0)

* Update to improve the code:

1. Added license.
2. Added more comments.
3. Wrap byte array instead of copy to make a ByteString.
4. Moved all reflection instantiation to static class loading time.

* Use LiteralByteString to wrap byte array instead of copying it.


Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-08 11:31:51 -07:00
huaxiangsun ed830222da
HBASE-24120 Flakey Test: TestReplicationAdminWithClusters timeout (#1441)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-08 09:28:51 -07:00
huaxiangsun d6451a8e2b
HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (#1433)
Signed-off-by: stack <stack@apache.org>
2020-04-07 20:36:11 -07:00
Michael Stack 80d4a09ddc HBASE-24128 [Flakey Tests] Add retry on thrift cmdline if client fails plus misc debug (#1442)
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAssignment.java
 Saw case where Master failed startup but it came out as an IOE so we
 did not trip the retry logic.

hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
 Add some debug and up timeouts. This test fails frequently for me
 locally.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/locking/TestEntityLocks.java
 Up the wait from 2x 200ms to 10x in case a pause on hardware or GC.
 This test fails locally and up on jenkins.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClearRegionBlockCache.java
 Debug. Have assert say what bad count was.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 Fails on occasion. Found count is off by a few. Tricky to debug. HBASE-24129 to reenable.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
 Debug. Add wait and check before moving to assert.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Check for null before shutting; can be null if failed start.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
 Add retry if client messes up connection. Fails for me locally.
2020-04-07 09:29:23 -07:00
Duo Zhang f60f0bdbf0
HBASE-24055 Make AsyncFSWAL can run on EC cluster (#1437)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-07 23:41:35 +08:00
binlijin 62718cdb28
HBASE-24077 When encounter RowTooBigException, log the row info. (#1379)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-07 10:28:35 +08:00
stack 0e30e00c56 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
Reapply but as an @Ignore for the flakey test.
2020-04-06 12:38:02 -07:00
stack 0dc2712c90 Revert "HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash"
Reverting in favor of adding an @Ignore on this test until
root cause of flakyness HBASE-24117 is addressed.

This reverts commit 9985c06647.
2020-04-06 12:35:52 -07:00
huaxiangsun da67491a56
HBASE-24105 [Flakey Test] regionserver.TestRegionReplicas (#1425)
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-05 13:20:23 -07:00