Commit Graph

7796 Commits

Author SHA1 Message Date
Duo Zhang e38531a70a HBASE-24314 Some classes still use log4j logger directly (#1637)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-05 20:25:10 +08:00
Nick Dimiduk fa1890c880 HBASE-24260 Add a ClusterManager that issues commands via coprocessor
Implements `ClusterManager` that relies on the new
`ShellExecEndpointCoprocessor` for remote shell command execution.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-04 10:50:42 -07:00
stack 5b823cf8bf HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
Addendum to address minor feedback on text
2020-05-04 08:25:35 -07:00
Michael Stack 616f277901 HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Fix weird brackets around each region name when logging.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
  Log when we hit the max merge limit. Also up limit to 64.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Make logs make more sense to operator.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Make RegionName show when you mouseover so long names don't mess up
 display of holes and overlaps.

Address Mingliang Liu liuml07 feedback

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-04 08:25:35 -07:00
niuyulin b2f625045a HBASE-24221 Support bulkLoadHFile by family (#1627)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(Cherry picked from commit 0a428e8b48)
2020-05-04 15:25:30 +01:00
stack b977d1f1dc HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Addendum; fix failing test
2020-05-01 17:51:54 -07:00
xincunSong 9d90287b6d [HBASE-24288]Allow admin user to create table and do bulkLoad (#1612)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-01 11:58:26 -07:00
gkanade 6147aebd00 HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Signed-off-by: stack <stack@apache.org>
2020-05-01 11:50:16 -07:00
bsglz 4262c72086
HBASE-22710 Wrong result in one case of scan that use raw and version… (#767)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-30 14:13:13 +05:30
xincunSong db49e28f16 HBASE-24282 'scanDetail' log message is missing when responseTooSlow happens on the first scan rpc call (#1604)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-29 16:06:01 -07:00
stack 595655f190 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately).
Addendum: Restore old semantic where merge of non-adjacent required
'force'
2020-04-29 14:36:52 -07:00
stack a020e461c5 HBASE-24215 [Flakey Tests] [ERROR] TestSecureRESTServer java.lang.NoClassDefFoundError: com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
Addendum: Add jersey-servlet to hadoop3 profile.

Made the hadoop3 profile in top-level pom same as it is for hadoop2
when it comes to exclusions. Then backed out previous attempt mostly.
Made the failing test medium-sized so it ran in its own jvm.
2020-04-29 07:37:05 -07:00
Duo Zhang bf2350e25f HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586)
Signed-off-by: stack <stack@apache.org>
2020-04-29 11:32:44 +08:00
Michael Stack 5d117afc77 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately) (#1570)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add new isOverlap method that takes list of RegionInfos checking that
 current RegionInfo is overlapped by the passed in Regions.

    Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-04-28 13:26:14 -07:00
Duo Zhang a0f3e23c78 Revert "HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)"
This breaks TestAccessController.testAccessControllerUserPermsRegexHandling

This reverts commit 1d1f481e72.
2020-04-27 22:57:32 +08:00
Duo Zhang 9c09ae5fa1 HBASE-24264 Disable TestNettyIPC.testHedgedAsyncEcho (#1589) 2020-04-27 09:43:15 +08:00
bsglz 40b07bf438
HBASE-23264 Resolve a TODO of BucketAllocator: "Why we add the extra … (#801)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-26 16:21:21 +05:30
Mohammad Arshad 1d1f481e72
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2020-04-25 17:46:35 +05:30
niuyulin 0cf67bbf82 HBASE-24222 remove FSUtils.checkAccess and replace with FileSystem.access in HBCK (#1557)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-23 23:59:42 +08:00
Viraj Jasani 5ad233b60c
HBASE-24024 : Reject multi() requests with rows higher than threshold (#1560)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-22 12:53:40 +05:30
Beata Sudi 1d35d0a9a9
HBASE-24139 : Balancer should avoid leaving idle region servers (#1511)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-04-22 10:12:03 +05:30
stack 07357a2f84 HBASE-24226 Address other hard references to '/tmp' found in Configuration 2020-04-21 16:02:04 -07:00
stack 5ce96fb565 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum #3; catch more '/tmp' references by hadoop.
Fix broken build.
2020-04-21 11:20:25 -07:00
stack 7e53c8929a HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum #3; catch more '/tmp' references by hadoop.
2020-04-21 09:43:12 -07:00
stack 671be53476 HBASE-24220 Allow that zk NOTEMPTY multi exception is retryable by running in-series
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java
 Cleanup checkstyle warnings. Don't depend on hbase-client
 ScannerCallable.

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
 Cut down on cluster resource usage.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/AbstractTestDLS.java
 Debug

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController3.java
 Debug

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
 Allow that NONEMPTY is retryable by running in series.
2020-04-20 15:44:08 -07:00
Guangxu Cheng 23358038d9 HBASE-23896 Snapshot owner cannot delete snapshot when ACL is enabled and Kerberos is not enabled (#1211)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-04-20 10:17:19 +08:00
Reid Chan 6354385a89 HBASE-24174 Fix findbugs warning on ServiceAuthorizationManager
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-18 13:53:57 +08:00
XinSun 0e9fcb418b HBASE-24166 Duplicate implementation for acquireLock between CreateTableProcedure and its parent class (#1486)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-18 10:33:04 +08:00
Joseph295 e1adf28390 HBASE-24182 log when the region is set to closing status (#1509)
Signed-off-by: stack <stack@apache.org>
2020-04-17 12:21:53 -07:00
Wei-Chiu Chuang b8b8e0afd4
HBASE-23833. The relocated hadoop-thirdparty protobuf breaks HBase asyncwal (#1301) (#1535)
* Use Reflection to access shaded Hadoop protobuf classes.

(cherry picked from commit a321e536989083ca3620bf2c53f12c07740bf5b0)

* Update to improve the code:

1. Added license.
2. Added more comments.
3. Wrap byte array instead of copy to make a ByteString.
4. Moved all reflection instantiation to static class loading time.

* Use LiteralByteString to wrap byte array instead of copying it.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 72727ff9be)
(cherry picked from commit ae3de38bed)
2020-04-17 12:08:10 -07:00
gkanade aa5b28a779 MetricsTable#updateFlushTime is wrong (#1539)
Authored-by: Gaurav Kanade <gkanade@microsoft.com>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-04-17 17:42:44 +05:30
Viraj Jasani 16e75698a9
HBASE-24195 : Admin.getRegionServers() should return live servers exc… (#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-16 20:17:04 +05:30
BukrosSzabolcs 319864cbaa
HBASE-24186: RegionMover ignores replicationId (#1512)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-16 19:58:02 +05:30
huaxiangsun 0edf6cf483
HBASE-24183 [flakey test] replication.TestAddToSerialReplicationPeer (#1514) (#1526)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 14:43:02 -07:00
stack e99d103bb5 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum: add test to check for '/tmp' references.
2020-04-15 10:37:55 -07:00
Reid Chan 28e729510d HBASE-24112 [RSGroup] Support renaming rsgroup (#1520)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 19:06:02 +08:00
GeorryHuang dc7fd7c32c HBASE-23994: Add WebUI to Canary (#1292)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-14 16:15:04 -07:00
binlijin a33e4d2168 HBASE-24181 Add region info when log meessages in HRegion. (#1506)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-14 20:01:16 +08:00
Jianfei Jiang 2e1a719e26 HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7 (#1453)
* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

Co-authored-by: 姜建飞 10222269 <jiang.jianfei@zte.com.cn>

Signed-off-by: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Signed-off-by: Norbert Kalmar <nkalmar@cloudera.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangdo@apache.org>
2020-04-13 13:49:12 -07:00
Michael Stack f2f86c1790 HBASE-24134 Down forked JVM heap size from 2800m to 2200m for jdk8 and jdk11 (#1451)
Down jdk8 forked jvm heap from 2800 to 2200 and the jdk11 heap from
3200 to 2200. Down the mvn size from 4G to 3.6G

Change how many puts done by TestMultiRespectsLimits because made
the test run the forked heap over 2.5G in size.

Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-13 12:05:20 -07:00
stack 7b2b6dcf97 HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded 2020-04-13 11:52:12 -07:00
stack 117164e12f HBASE-24167 [Flakey Tests] TestHRegionWithInMemoryFlush#testWritesWhileScanning gets stuck MVCC 2020-04-13 11:44:44 -07:00
stack f6ab525661 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Overwrite yarn configs that write /tmp dir by default.
2020-04-13 11:30:39 -07:00
Viraj Jasani cd259d7364
HBASE-24140 : Move CandidateGenerator and their implementors out of StochasticLoadBalancer (#1458)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-12 19:26:18 +05:30
huaxiangsun 5e11ff100a
HBASE-24161 [flakey test] locking.TestEntityLocks.testEntityLockTimeout (#1477) (#1481)
Signed-off-by: stack <stack@apache.org>
2020-04-10 09:40:50 -07:00
huaxiangsun eae7523cef
HBASE-24159 [flakey test] regionserver.TestRegionMergeTransactionOnCluster (#1472) (#1475)
Delete tables after each test cases to avoid polluting states.

Signed-off-by: stack <stack@apache.org>
2020-04-09 21:08:10 -07:00
Pankaj 8fa93ef858 HBASE-24074 ConcurrentModificationException occured in ReplicationSourceManager while refreshing the peer (#1471)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-09 16:42:46 -07:00
Reid Chan 436567e34d HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be (#1439)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-04-09 16:49:48 +08:00
huaxiangsun 5b1f908027
HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (#1433) (#1459)
Signed-off-by: stack <stack@apache.org>
2020-04-08 14:11:08 -07:00
Sean Busbey 0822b6d167 HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer (#1455)
- at DEBUG log messages about RegionCountSkewCostFunction region/server totals
- at DEBUG log messages about the decision to balance or not with total costs
- at TRACE log messages about region count on each server RegionCountSkewCostFunction sees
- at TRACE log message with the individual cost functions used in the decision to balance or not

Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 2d78a286b6)
2020-04-08 15:49:29 -05:00