Commit Graph

7900 Commits

Author SHA1 Message Date
huaxiangsun 2b2e6c7789
HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished (#1672) (#1678)
Signed-off-by: Huaxiang Sun<huaxiangsun@apache.org>
2020-05-07 14:11:44 -07:00
stack 93ca07971c HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP #1669
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-07 13:53:43 -07:00
stack 378e6ed127 HBASE-24342 [Flakey Tests] Disable TestClusterPortAssignment.testClusterPortAssignment as it can't pass 100% of the time 2020-05-07 13:21:51 -07:00
huaxiangsun f49fc24ecf
HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent (#1660) (#1663)
It addresses couple issues:
   1. Make sure deleteMergeQualifiers() does not delete the row if there is no columns with "merge" keyword.
   2. GCMulitpleMergedRegionsProcedure now acquire an exclusive lock on the child region.

Signed-off-by: stack <stack@apache.org>
2020-05-06 10:01:25 -07:00
Michael Stack 1e8d3524f0 HBASE-24331 [Flakey Test] TestJMXListener rmi port clash (#1657)
Add check that we can make jmx connector in setup. If we can't retry.
2020-05-06 09:29:21 -07:00
Reid Chan ead9509d31 HBASE-24336 [Metrics] FSDataInputStream's localBytesRead is wrong (#1667)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-06 19:38:37 +08:00
Mohammad Arshad 20c2f2898c
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1631)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-06 15:18:16 +05:30
Duo Zhang b2f4e6ecf6 HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 16:20:03 +08:00
huaxiangsun e6fe01a88c
HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles (#1613) (#1662)
Signed-off-by: stack <stack@apache.org>
2020-05-05 23:01:29 -07:00
Nick Dimiduk d28b56f111 HBASE-23976 Use nio for file manipulation in TestVerifyBucketCacheFile (#1342)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-05-05 22:05:49 -07:00
Michael Stack 7c92e145b9 HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656)
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
2020-05-05 21:36:20 -07:00
stack db3d42bd9a HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
Addendum; add back bit missing off end of test.
2020-05-05 21:16:09 -07:00
Michael Stack bee66bd22a HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:28:31 -07:00
Duo Zhang e38531a70a HBASE-24314 Some classes still use log4j logger directly (#1637)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-05 20:25:10 +08:00
Nick Dimiduk fa1890c880 HBASE-24260 Add a ClusterManager that issues commands via coprocessor
Implements `ClusterManager` that relies on the new
`ShellExecEndpointCoprocessor` for remote shell command execution.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-04 10:50:42 -07:00
stack 5b823cf8bf HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
Addendum to address minor feedback on text
2020-05-04 08:25:35 -07:00
Michael Stack 616f277901 HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Fix weird brackets around each region name when logging.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
  Log when we hit the max merge limit. Also up limit to 64.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Make logs make more sense to operator.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Make RegionName show when you mouseover so long names don't mess up
 display of holes and overlaps.

Address Mingliang Liu liuml07 feedback

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-04 08:25:35 -07:00
niuyulin b2f625045a HBASE-24221 Support bulkLoadHFile by family (#1627)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(Cherry picked from commit 0a428e8b48)
2020-05-04 15:25:30 +01:00
stack b977d1f1dc HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Addendum; fix failing test
2020-05-01 17:51:54 -07:00
xincunSong 9d90287b6d [HBASE-24288]Allow admin user to create table and do bulkLoad (#1612)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-01 11:58:26 -07:00
gkanade 6147aebd00 HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Signed-off-by: stack <stack@apache.org>
2020-05-01 11:50:16 -07:00
bsglz 4262c72086
HBASE-22710 Wrong result in one case of scan that use raw and version… (#767)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-30 14:13:13 +05:30
xincunSong db49e28f16 HBASE-24282 'scanDetail' log message is missing when responseTooSlow happens on the first scan rpc call (#1604)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-29 16:06:01 -07:00
stack 595655f190 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately).
Addendum: Restore old semantic where merge of non-adjacent required
'force'
2020-04-29 14:36:52 -07:00
stack a020e461c5 HBASE-24215 [Flakey Tests] [ERROR] TestSecureRESTServer java.lang.NoClassDefFoundError: com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
Addendum: Add jersey-servlet to hadoop3 profile.

Made the hadoop3 profile in top-level pom same as it is for hadoop2
when it comes to exclusions. Then backed out previous attempt mostly.
Made the failing test medium-sized so it ran in its own jvm.
2020-04-29 07:37:05 -07:00
Duo Zhang bf2350e25f HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586)
Signed-off-by: stack <stack@apache.org>
2020-04-29 11:32:44 +08:00
Michael Stack 5d117afc77 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately) (#1570)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add new isOverlap method that takes list of RegionInfos checking that
 current RegionInfo is overlapped by the passed in Regions.

    Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-04-28 13:26:14 -07:00
Duo Zhang a0f3e23c78 Revert "HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)"
This breaks TestAccessController.testAccessControllerUserPermsRegexHandling

This reverts commit 1d1f481e72.
2020-04-27 22:57:32 +08:00
Duo Zhang 9c09ae5fa1 HBASE-24264 Disable TestNettyIPC.testHedgedAsyncEcho (#1589) 2020-04-27 09:43:15 +08:00
bsglz 40b07bf438
HBASE-23264 Resolve a TODO of BucketAllocator: "Why we add the extra … (#801)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-26 16:21:21 +05:30
Mohammad Arshad 1d1f481e72
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2020-04-25 17:46:35 +05:30
niuyulin 0cf67bbf82 HBASE-24222 remove FSUtils.checkAccess and replace with FileSystem.access in HBCK (#1557)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-23 23:59:42 +08:00
Viraj Jasani 5ad233b60c
HBASE-24024 : Reject multi() requests with rows higher than threshold (#1560)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-22 12:53:40 +05:30
Beata Sudi 1d35d0a9a9
HBASE-24139 : Balancer should avoid leaving idle region servers (#1511)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-04-22 10:12:03 +05:30
stack 07357a2f84 HBASE-24226 Address other hard references to '/tmp' found in Configuration 2020-04-21 16:02:04 -07:00
stack 5ce96fb565 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum #3; catch more '/tmp' references by hadoop.
Fix broken build.
2020-04-21 11:20:25 -07:00
stack 7e53c8929a HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum #3; catch more '/tmp' references by hadoop.
2020-04-21 09:43:12 -07:00
stack 671be53476 HBASE-24220 Allow that zk NOTEMPTY multi exception is retryable by running in-series
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java
 Cleanup checkstyle warnings. Don't depend on hbase-client
 ScannerCallable.

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
 Cut down on cluster resource usage.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/AbstractTestDLS.java
 Debug

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController3.java
 Debug

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
 Allow that NONEMPTY is retryable by running in series.
2020-04-20 15:44:08 -07:00
Guangxu Cheng 23358038d9 HBASE-23896 Snapshot owner cannot delete snapshot when ACL is enabled and Kerberos is not enabled (#1211)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-04-20 10:17:19 +08:00
Reid Chan 6354385a89 HBASE-24174 Fix findbugs warning on ServiceAuthorizationManager
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-18 13:53:57 +08:00
XinSun 0e9fcb418b HBASE-24166 Duplicate implementation for acquireLock between CreateTableProcedure and its parent class (#1486)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-18 10:33:04 +08:00
Joseph295 e1adf28390 HBASE-24182 log when the region is set to closing status (#1509)
Signed-off-by: stack <stack@apache.org>
2020-04-17 12:21:53 -07:00
Wei-Chiu Chuang b8b8e0afd4
HBASE-23833. The relocated hadoop-thirdparty protobuf breaks HBase asyncwal (#1301) (#1535)
* Use Reflection to access shaded Hadoop protobuf classes.

(cherry picked from commit a321e536989083ca3620bf2c53f12c07740bf5b0)

* Update to improve the code:

1. Added license.
2. Added more comments.
3. Wrap byte array instead of copy to make a ByteString.
4. Moved all reflection instantiation to static class loading time.

* Use LiteralByteString to wrap byte array instead of copying it.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 72727ff9be)
(cherry picked from commit ae3de38bed)
2020-04-17 12:08:10 -07:00
gkanade aa5b28a779 MetricsTable#updateFlushTime is wrong (#1539)
Authored-by: Gaurav Kanade <gkanade@microsoft.com>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-04-17 17:42:44 +05:30
Viraj Jasani 16e75698a9
HBASE-24195 : Admin.getRegionServers() should return live servers exc… (#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-16 20:17:04 +05:30
BukrosSzabolcs 319864cbaa
HBASE-24186: RegionMover ignores replicationId (#1512)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-16 19:58:02 +05:30
huaxiangsun 0edf6cf483
HBASE-24183 [flakey test] replication.TestAddToSerialReplicationPeer (#1514) (#1526)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 14:43:02 -07:00
stack e99d103bb5 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum: add test to check for '/tmp' references.
2020-04-15 10:37:55 -07:00
Reid Chan 28e729510d HBASE-24112 [RSGroup] Support renaming rsgroup (#1520)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 19:06:02 +08:00
GeorryHuang dc7fd7c32c HBASE-23994: Add WebUI to Canary (#1292)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-14 16:15:04 -07:00