Commit Graph

16947 Commits

Author SHA1 Message Date
Duo Zhang 388fae73ff HBASE-24333 Backport HBASE-24304 "Separate a hbase-asyncfs module" to branch-2.x (#1664)
Signed-off-by: stack <stack@apache.org>
2020-05-09 12:24:45 +08:00
stack 82fde05f65 HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625)
Addendum to fix failing MR tests; e.g. TestExportSnapshot.
2020-05-08 13:04:23 -07:00
Duo Zhang 0f1b9d0811 HBASE-24310 Use Slf4jRequestLog for hbase-http (#1634)
Signed-off-by: stack <stack@apache.org>
2020-05-08 11:22:24 +08:00
huaxiangsun 2b2e6c7789
HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished (#1672) (#1678)
Signed-off-by: Huaxiang Sun<huaxiangsun@apache.org>
2020-05-07 14:11:44 -07:00
stack 93ca07971c HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP #1669
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-07 13:53:43 -07:00
Nick Dimiduk 6cf140850c HBASE-24295 [Chaos Monkey] abstract logging through the class hierarchy ; ADDENDUM
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-07 13:27:29 -07:00
stack 378e6ed127 HBASE-24342 [Flakey Tests] Disable TestClusterPortAssignment.testClusterPortAssignment as it can't pass 100% of the time 2020-05-07 13:21:51 -07:00
bsglz 9dd8c9607c HBASE-24335 Support deleteall with ts but without column in shell mode (#1668)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 2cafe81e9c)
2020-05-07 18:25:43 +01:00
huaxiangsun f49fc24ecf
HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent (#1660) (#1663)
It addresses couple issues:
   1. Make sure deleteMergeQualifiers() does not delete the row if there is no columns with "merge" keyword.
   2. GCMulitpleMergedRegionsProcedure now acquire an exclusive lock on the child region.

Signed-off-by: stack <stack@apache.org>
2020-05-06 10:01:25 -07:00
Michael Stack 1e8d3524f0 HBASE-24331 [Flakey Test] TestJMXListener rmi port clash (#1657)
Add check that we can make jmx connector in setup. If we can't retry.
2020-05-06 09:29:21 -07:00
Reid Chan ead9509d31 HBASE-24336 [Metrics] FSDataInputStream's localBytesRead is wrong (#1667)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-06 19:38:37 +08:00
Mohammad Arshad 20c2f2898c
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1631)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-06 15:18:16 +05:30
Duo Zhang b2f4e6ecf6 HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 16:20:03 +08:00
huaxiangsun e6fe01a88c
HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles (#1613) (#1662)
Signed-off-by: stack <stack@apache.org>
2020-05-05 23:01:29 -07:00
Nick Dimiduk d28b56f111 HBASE-23976 Use nio for file manipulation in TestVerifyBucketCacheFile (#1342)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-05-05 22:05:49 -07:00
Michael Stack 7c92e145b9 HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656)
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
2020-05-05 21:36:20 -07:00
stack db3d42bd9a HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
Addendum; add back bit missing off end of test.
2020-05-05 21:16:09 -07:00
Michael Stack bee66bd22a HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:28:31 -07:00
Andrew Purtell ebe6b94a6c
HBASE-24322 UnsafeAvailChecker should also check that required methods are available (#1650)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-05 12:11:46 -07:00
Duo Zhang e38531a70a HBASE-24314 Some classes still use log4j logger directly (#1637)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-05 20:25:10 +08:00
Wei-Chiu Chuang df95e0ea4c HBASE-24258 [Hadoop3.3] Update license for org.ow2.asm:* (#1582)
Signed-off-by: stack <stack@apache.org>
2020-05-04 14:58:39 -07:00
Nick Dimiduk f6ee3163bd HBASE-24295 [Chaos Monkey] abstract logging through the class hierarchy
Adds `protected abstract Logger getLogger()` to `Action` so that
implementation's names are logged when actions are performed.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-04 11:36:46 -07:00
Nick Dimiduk fa1890c880 HBASE-24260 Add a ClusterManager that issues commands via coprocessor
Implements `ClusterManager` that relies on the new
`ShellExecEndpointCoprocessor` for remote shell command execution.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-04 10:50:42 -07:00
Michael Stack 527b14f4f1 HBASE-24303 Undo core of parent TestSecureRESTServer change; use fix over in HBASE-24280 instead (#1630) 2020-05-04 10:13:42 -07:00
stack 5b823cf8bf HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
Addendum to address minor feedback on text
2020-05-04 08:25:35 -07:00
Michael Stack 616f277901 HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Fix weird brackets around each region name when logging.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
  Log when we hit the max merge limit. Also up limit to 64.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Make logs make more sense to operator.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Make RegionName show when you mouseover so long names don't mess up
 display of holes and overlaps.

Address Mingliang Liu liuml07 feedback

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-04 08:25:35 -07:00
niuyulin b2f625045a HBASE-24221 Support bulkLoadHFile by family (#1627)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(Cherry picked from commit 0a428e8b48)
2020-05-04 15:25:30 +01:00
Wellington Ramos Chevreuil 4da8209a4d HBASE-24302 Add an "ignoreTimestamps" option (defaulted to false) to … (#1623)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry-picked from commit 3d59e328c8)
2020-05-04 15:09:38 +01:00
Jan Hentschel d8738ae054
HBASE-24301 Updated Apache POM to version 23
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-02 14:45:57 +02:00
stack b977d1f1dc HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Addendum; fix failing test
2020-05-01 17:51:54 -07:00
xincunSong 9d90287b6d [HBASE-24288]Allow admin user to create table and do bulkLoad (#1612)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-01 11:58:26 -07:00
gkanade 6147aebd00 HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Signed-off-by: stack <stack@apache.org>
2020-05-01 11:50:16 -07:00
Michael Stack a3c8ec8a7b HBASE-24294 [Flakey Tests] TestThriftHttpServer BindException (#1619)
Refactor Thrift tests so resilient when launch of ThriftServer runs
into BindException.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestBindExceptionHandling.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThrift2HttpServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThrift2ServerCmdLine.java
 Refactor to use new ThriftServerRunner and ThriftServerSupplier

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
 Log list of existing tables on assert.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftSpnegoHttpFallbackServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftSpnegoHttpServer.java
 Add Ignore for testRunThriftServerWithHeaderBufferLength. Its tested in
 superclass. No need to test again here. Besides, its destructive test
 leaving behind dregs that mess up the retry.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
 Utility. Thread to host ThriftServer and exception if one after run
 completes.
2020-05-01 08:40:09 -07:00
bsglz 4262c72086
HBASE-22710 Wrong result in one case of scan that use raw and version… (#767)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-30 14:13:13 +05:30
Peter Somogyi 3cd5eb1796 HBASE-24285 Move to hbase-thirdparty-3.3.0 (#1605)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-30 10:11:54 +02:00
xincunSong db49e28f16 HBASE-24282 'scanDetail' log message is missing when responseTooSlow happens on the first scan rpc call (#1604)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-29 16:06:01 -07:00
stack 595655f190 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately).
Addendum: Restore old semantic where merge of non-adjacent required
'force'
2020-04-29 14:36:52 -07:00
Nick Dimiduk bd27542a45 HBASE-24274 `RESTApiClusterManager` attempts to deserialize response using serialization API
Use the correct GSON API for deserializing service responses. Add
simple unit test covering a very limited selection of the overall API
surface area, just enough to ensure deserialization works.

Signed-off-by: stack <stack@apache.org>
2020-04-29 13:09:03 -07:00
stack a020e461c5 HBASE-24215 [Flakey Tests] [ERROR] TestSecureRESTServer java.lang.NoClassDefFoundError: com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
Addendum: Add jersey-servlet to hadoop3 profile.

Made the hadoop3 profile in top-level pom same as it is for hadoop2
when it comes to exclusions. Then backed out previous attempt mostly.
Made the failing test medium-sized so it ran in its own jvm.
2020-04-29 07:37:05 -07:00
Duo Zhang 4138813c34 HBASE-24281 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:26:06 +08:00
Duo Zhang 68771777ec Revert "HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)"
This reverts commit 8adaa56caa.
2020-04-29 18:25:54 +08:00
Duo Zhang 8adaa56caa HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:11:17 +08:00
Duo Zhang bf2350e25f HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586)
Signed-off-by: stack <stack@apache.org>
2020-04-29 11:32:44 +08:00
Michael Stack 5d117afc77 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately) (#1570)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add new isOverlap method that takes list of RegionInfos checking that
 current RegionInfo is overlapped by the passed in Regions.

    Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-04-28 13:26:14 -07:00
Josh Elser c78bbbcdcb HBASE-24252 Implement proxyuser/doAs mechanism for hbase-http
copy ProxyUserAuthenticationFilter from Hadoop
add hbase.security.authentication.spnego.kerberos.proxyuser.enable parameter (default false)
wire ProxyUserAuthenticationFilter into HttpServer

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-27 13:54:14 -04:00
Nick Dimiduk 77a5f913fa HBASE-24143 [JDK11] Switch default garbage collector from CMS ; Addendum (#1594)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-27 09:39:05 -07:00
Duo Zhang a0f3e23c78 Revert "HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)"
This breaks TestAccessController.testAccessControllerUserPermsRegexHandling

This reverts commit 1d1f481e72.
2020-04-27 22:57:32 +08:00
Duo Zhang 9c09ae5fa1 HBASE-24264 Disable TestNettyIPC.testHedgedAsyncEcho (#1589) 2020-04-27 09:43:15 +08:00
bsglz 40b07bf438
HBASE-23264 Resolve a TODO of BucketAllocator: "Why we add the extra … (#801)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-26 16:21:21 +05:30
Mohammad Arshad 1d1f481e72
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2020-04-25 17:46:35 +05:30