Commit Graph

17196 Commits

Author SHA1 Message Date
stack 99e96cca3b HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
Addendum; add back bit missing off end of test.
2020-05-05 21:15:22 -07:00
Michael Stack 528e9d6454
HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656)
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
2020-05-05 21:08:09 -07:00
Michael Stack 80561504d2
HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:27:11 -07:00
Andrew Purtell 5dc10dbba5
HBASE-24322 UnsafeAvailChecker should also check that required methods are available (#1650)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-05 12:11:37 -07:00
Duo Zhang 64bfc99f05 HBASE-24314 Some classes still use log4j logger directly (#1637)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-05 20:24:58 +08:00
Anoop Sam John 5e5b13cb51
HBASE-24311 Add more details in MultiVersionConcurrencyControl STUCK log message (#1654) 2020-05-05 16:25:40 +05:30
Wei-Chiu Chuang 3a375e668c HBASE-24258 [Hadoop3.3] Update license for org.ow2.asm:* (#1582)
Signed-off-by: stack <stack@apache.org>
2020-05-04 14:58:15 -07:00
Nick Dimiduk 9cf541bc8d HBASE-24295 [Chaos Monkey] abstract logging through the class hierarchy
Adds `protected abstract Logger getLogger()` to `Action` so that
implementation's names are logged when actions are performed.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-04 11:41:33 -07:00
Nick Dimiduk 47dca8eb45 HBASE-24260 Add a ClusterManager that issues commands via coprocessor
Implements `ClusterManager` that relies on the new
`ShellExecEndpointCoprocessor` for remote shell command execution.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-04 10:52:28 -07:00
Michael Stack 337529ba2c
HBASE-24303 Undo core of parent TestSecureRESTServer change; use fix over in HBASE-24280 instead (#1630) 2020-05-04 10:12:32 -07:00
stack ffae8a570c HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
Addendum to address minor feedback on text
2020-05-04 08:24:09 -07:00
Michael Stack 1dd4b13e17
HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Fix weird brackets around each region name when logging.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
  Log when we hit the max merge limit. Also up limit to 64.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Make logs make more sense to operator.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Make RegionName show when you mouseover so long names don't mess up
 display of holes and overlaps.

Address Mingliang Liu liuml07 feedback

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-04 08:21:52 -07:00
Wellington Ramos Chevreuil c6be5b0916 HBASE-24302 Add an "ignoreTimestamps" option (defaulted to false) to … (#1623)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit 3d59e328c8)
2020-05-04 11:29:12 +01:00
niuyulin 0a428e8b48
HBASE-24221 Support bulkLoadHFile by family (#1627)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-05-03 13:46:53 +01:00
Jan Hentschel 13c4030cfc
HBASE-24301 Updated Apache POM to version 23
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-02 14:38:57 +02:00
stack da92a12bf5 HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Addendum; fix failing test
2020-05-01 17:31:59 -07:00
xincunSong 0d2b959d20 [HBASE-24288]Allow admin user to create table and do bulkLoad (#1612)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-01 11:58:06 -07:00
gkanade 1df3bee5f6 HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616)
Signed-off-by: stack <stack@apache.org>
2020-05-01 11:49:32 -07:00
Michael Stack 4cd893ad47
HBASE-24294 [Flakey Tests] TestThriftHttpServer BindException (#1619)
Refactor Thrift tests so resilient when launch of ThriftServer runs
into BindException.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestBindExceptionHandling.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThrift2HttpServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThrift2ServerCmdLine.java
 Refactor to use new ThriftServerRunner and ThriftServerSupplier

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
 Log list of existing tables on assert.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftSpnegoHttpFallbackServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftSpnegoHttpServer.java
 Add Ignore for testRunThriftServerWithHeaderBufferLength. Its tested in
 superclass. No need to test again here. Besides, its destructive test
 leaving behind dregs that mess up the retry.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
 Utility. Thread to host ThriftServer and exception if one after run
 completes.
2020-05-01 08:38:44 -07:00
bsglz a6aea729c5
HBASE-22710 Wrong result in one case of scan that use raw and version… (#767)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-30 14:11:04 +05:30
Peter Somogyi ccef8d40f4 HBASE-24285 Move to hbase-thirdparty-3.3.0 (#1605)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-30 10:08:24 +02:00
stack a499f3748d Revert "HBASE-24285 Move to hbase-thirdparty-3.3.0"
Premature application. Revert.

This reverts commit 04c5ea13da.
2020-04-29 16:06:34 -07:00
xincunSong d59af05b0f HBASE-24282 'scanDetail' log message is missing when responseTooSlow happens on the first scan rpc call (#1604)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-29 16:04:40 -07:00
Peter Somogyi 04c5ea13da HBASE-24285 Move to hbase-thirdparty-3.3.0 2020-04-29 15:53:16 -07:00
stack 93f0d4ce06 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately).
Addendum: Restore old semantic where merge of non-adjacent required
'force'
2020-04-29 14:34:37 -07:00
Nick Dimiduk fdf2bd7312 HBASE-24274 `RESTApiClusterManager` attempts to deserialize response using serialization API
Use the correct GSON API for deserializing service responses. Add
simple unit test covering a very limited selection of the overall API
surface area, just enough to ensure deserialization works.

Signed-off-by: stack <stack@apache.org>
2020-04-29 13:13:14 -07:00
Duo Zhang 1f556b3ad9 HBASE-24281 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:25:41 +08:00
Duo Zhang 379b48bb5d Revert "HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)"
This reverts commit dc9608ec0b.
2020-04-29 18:25:23 +08:00
Duo Zhang dc9608ec0b HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:08:33 +08:00
Duo Zhang 922921ee5f HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586)
Signed-off-by: stack <stack@apache.org>
2020-04-29 11:31:32 +08:00
stack 3b0812d3e1 HBASE-24215 [Flakey Tests] [ERROR] TestSecureRESTServer java.lang.NoClassDefFoundError: com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
Addendum #2: Add jersey-servlet to hadoop3 profile.
2020-04-28 20:18:31 -07:00
Michael Stack 9e697d0ea6
HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately) (#1570)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add new isOverlap method that takes list of RegionInfos checking that
 current RegionInfo is overlapped by the passed in Regions.

    Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-04-28 13:22:00 -07:00
Josh Elser 8eb2f6752c HBASE-24252 Implement proxyuser/doAs mechanism for hbase-http
copy ProxyUserAuthenticationFilter from Hadoop
add hbase.security.authentication.spnego.kerberos.proxyuser.enable parameter (default false)
wire ProxyUserAuthenticationFilter into HttpServer

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-27 13:15:34 -04:00
Nick Dimiduk 318af8d6f8 HBASE-24143 [JDK11] Switch default garbage collector from CMS ; Addendum (#1594)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-27 09:36:35 -07:00
Duo Zhang 47348f5887 Revert "HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)"
This breaks TestAccessController.testAccessControllerUserPermsRegexHandling

This reverts commit 66e90dece2.
2020-04-27 22:50:02 +08:00
Duo Zhang 5af807153a HBASE-24264 Disable TestNettyIPC.testHedgedAsyncEcho (#1589) 2020-04-27 09:43:09 +08:00
bsglz ee5283ee82
HBASE-23264 Resolve a TODO of BucketAllocator: "Why we add the extra … (#801)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-26 16:20:40 +05:30
Mohammad Arshad 66e90dece2
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2020-04-25 17:45:32 +05:30
Peter Somogyi b2e9321d97 HBASE-24251 PreCommit build is failing because of can not build docker image; addendum (#1579)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-24 13:59:51 +02:00
Duo Zhang 4f0d05b462 HBASE-24253 HBASE-Find-Flaky-Tests job is failing because of can not build docker image (#1578)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-24 19:47:12 +08:00
Duo Zhang a0295662da HBASE-24251 PreCommit build is failing because of can not build docker image (#1575)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-24 16:23:34 +08:00
stack da92d016c4 HBASE-24215 [Flakey Tests] [ERROR] TestSecureRESTServer java.lang.NoClassDefFoundError: com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate
Second attempt. Made the hadoop3 profile in top-level pom same as it is
for hadoop2 when it comes to exclusions. Then backed out previous
  attempt mostly. Made the failing test medium-sized so it ran in its
  own jvm.
2020-04-23 22:22:35 -07:00
niuyulin 90d738107f HBASE-24222 remove FSUtils.checkAccess and replace with FileSystem.access in HBCK (#1557)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-23 23:59:34 +08:00
Viraj Jasani b40381857a
HBASE-24024 : Reject multi() requests with rows higher than threshold (#1560)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-22 12:51:49 +05:30
Beata Sudi 74c033db83
HBASE-24139 : Balancer should avoid leaving idle region servers (#1511)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-04-22 10:11:16 +05:30
Duo Zhang 15a46388f1 HBASE-24218 Add hadoop 3.2.x in hadoop check (#1559)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-22 10:52:53 +08:00
Duo Zhang 6e95575669
HBASE-24213 Backport the pre commit changes for HBASE-24169 to all branches (#1548)
Signed-off-by: stack <stack@apache.org>
2020-04-22 10:26:10 +08:00
stack f8ce21f096 HBASE-24226 Address other hard references to '/tmp' found in Configuration 2020-04-21 17:39:30 -07:00
stack 3c1725f2f0 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException Addendum #3; catch more '/tmp' references by hadoop. Fix broken build. 2020-04-21 17:39:20 -07:00
stack b262b9a013 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException Addendum #3; catch more '/tmp' references by hadoop. 2020-04-21 17:39:04 -07:00