Commit Graph

17540 Commits

Author SHA1 Message Date
Huaxiang Sun 542244056c
HBASE-23957 [flakey test] client.TestMultiParallel fails to read hbase-site.xml (#1310)
Signed-off-by: Nick Dimiduk ndimiduk@apache.org
Signed-off-by: stack <stack@apache.org>
2020-03-22 17:43:04 -07:00
Guanghao Zhang 2a60a61a73
HBASE-24033 Add ut for loading the corrupt recovered hfiles (#1322)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-22 22:56:30 +08:00
niuyulin d4ab645531
HBASE-23949 refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced (#1279)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-03-22 21:54:57 +08:00
Pankaj ca533a77cb
HBASE-23633 Find a way to handle the corrupt recovered hfiles (#1233)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-03-22 16:39:56 +08:00
Toshihiro Suzuki 9b4e75a7f6
HBASE-24030 Add necessary validations to HRegion.checkAndMutate() and HRegion.checkAndRowMutate() (#1315)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-22 11:55:52 +09:00
Toshihiro Suzuki b53fa3c51f
HBASE-24031 TestHRegion.testCheckAndMutate_WithFilters is flaky (#1316)
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2020-03-22 05:04:16 +09:00
Michael Stack 5592d0c191 HBASE-23984 [Flakey Tests] TestMasterAbortAndRSGotKilled fails in teardown (#1311)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Change parameter name and add javadoc to make it more clear what the
 param actually is.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/AssignRegionHandler.java
 Move postOpenDeployTasks so if it fails to talk to the Master -- which
 can happen on cluster shutdown -- then we will do cleanup of state;
 without this the RS can get stuck and won't go down.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
 Add handleException so CRH looks more like UnassignRegionHandler and
 AssignRegionHandler around exception handling. Add a bit of doc on
 why CRH.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/UnassignRegionHandler.java
 Right shift most of the body of process so can add in a finally
 that cleans up rs.getRegionsInTransitionInRS is on exception
 (otherwise outstanding entries can stop a RS going down on cluster
 shutdown)

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-20 15:26:58 -07:00
Guanghao Zhang 080d8645ed
HBASE-24023 Add 2.2.4 to download page (#1314)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-20 23:57:19 +08:00
Viraj Jasani 44b3ac6457
HBASE-23977 : Resolve flakes present in TestSlowLogRecorder (ADDENDUM) 2020-03-20 13:27:06 +05:30
Viraj Jasani 26b31e351c
HBASE-23977 : Resolve flakes present in TestSlowLogRecorder (#1286)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-03-19 15:35:49 +05:30
Huaxiang Sun 52a04e69c1
HBASE-23999 [flakey test] TestTableOutputFormatConnectionExhaust (#1302)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-03-18 21:51:04 -07:00
stack de967502ab HBASE-24017 Turn down flakey rerun rate on all but hot branches
On master branch run twice a day instead of every 4 hours.
2020-03-18 15:50:39 -07:00
stack add2d2ac57 HBASE-24016 Change nightly poll from cron @daily to pollSCM @daily; i.e. run nightly if a change ONLY 2020-03-18 14:39:53 -07:00
Jan Hentschel d7e70bb497
HBASE-23943 Updated Rubocop configuration to current version
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-18 11:09:36 +01:00
Michael Stack 3e1cf00c71 HBASE-23993 Use loopback for zk standalone server in minizkcluster (#1291)
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Have client and server use loopback instead of 'localhost'

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-17 20:26:06 -07:00
Duo Zhang 9190a6b59a
HBASE-23989 Consider 2.10.0 in our support matrix (#1290)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-18 09:58:16 +08:00
stack 9804f73c66 HBASE-23971 protoc warns: "no protobuf syntax specified" (#1278)
Remove overcommit in TestUsersOperationsWithSecureHadoop
(Noticed by ndimiduk)
2020-03-17 15:28:26 -07:00
Nick Dimiduk 2e1485f505 HBASE-24004 Include hadoop version in Nightly report name 2020-03-17 14:49:59 -07:00
Wei-Chiu Chuang 60de4c2e31
HBASE-22103. HDFS-13209 in Hadoop 3.3.0 breaks asyncwal. (#1284)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-17 14:36:56 +08:00
Duo Zhang 795e9887b5
HBASE-23986 Bump hadoop-two.version to 2.10.0 on master and branch-2 (#1289)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-03-17 08:05:56 +08:00
Nick Dimiduk 059c189451 HBASE-23829 Get `-PrunSmallTests` passing on JDK11
Signed-off-by: stack <stack@apache.org>
2020-03-16 15:21:37 -07:00
stack f66cbe1a40 HBASE-23992 Fix TestAdminShell and TestQuotasShell mistakenly broken by parent commit 2020-03-14 15:18:21 -07:00
stack eefeeceedf HBASE-23991 [Flakey Tests] Disable TestSecureExport.testVisibilityLabels 2020-03-14 14:05:36 -07:00
Duo Zhang ba3f02629a
HBASE-23987 NettyRpcClientConfigHelper will not share event loop by default which is incorrect (#1288)
Signed-off-by: stack <stack@apache.org>
2020-03-14 13:47:41 +08:00
xuqinya1 b9e74e0b0a
HBASE-23967 Improve the accuracy of the method sizeToString (#1273)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-13 19:41:48 +05:30
Jan Hentschel 1b163d98b9
HBASE-23847 Removed deprecated setStartRow from Scan (#1220)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-13 08:43:17 +08:00
Duo Zhang e5b0455c38 HBASE-23077 Addendum typo fix fingbugs -> findbugs 2020-03-13 08:36:39 +08:00
stack 02b6bbe7ce HBASE-23979 Disable TestSlowLogRecorder until parent addressed 2020-03-12 17:24:21 -07:00
Nick Dimiduk c088d2edc9 HBASE-23978 Dockerfiles reusing stale apt sources info
Signed-off-by: stack <stack@apache.org>
2020-03-12 17:09:06 -07:00
WenFeiYi abf0a22259
HBASE-23971 protoc warns: "no protobuf syntax specified" (#1278)
Signed-off-by: stack <stack@apache.org>
2020-03-12 08:32:22 -07:00
stack 26f3702735 HBASE-23974 [Flakey Tests] Allow that server may not yet be cleared from DeadServers in TestHBCKSCP 2020-03-11 23:04:16 -07:00
stack d42e9768d0 HBASE-23956 Use less resources running tests (#1266)
Addendum: TestNamespaceCommands was using up the limit on
nioeventloopgroup.... so create them per run of client.
2020-03-11 22:48:22 -07:00
stack 20658c5139 HBASE-23956 Use less resources running tests (#1266)
Addendum: Two more tests that had issues because parent cut
down the number of default RegionServers launched.
2020-03-11 21:03:10 -07:00
Duo Zhang f4c576582b HBASE-23077 Addendum remove findbugs annotations dependency from shaded client 2020-03-12 11:41:32 +08:00
stack 4b64558ebc HBASE-23956 Use less resources running tests (#1266)
Addendum: This test was depending on hard-coded number for
log cleaners; the parent patch made them less in test context.
2020-03-11 20:34:29 -07:00
Duo Zhang c5aa2bc121
HBASE-23077 move entirely to spotbugs (#1265)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-03-12 10:18:09 +08:00
stack cf121227aa HBASE-23930 Shell should attempt to format `timestamp` attributes as ISO-8601
Make display of Cell timestamp be ISO8601 format instead of pure milliseconds.
2020-03-11 17:15:15 -07:00
Minwoo Kang 6f1e8abc30
HBASE-23561 acquire lock before update regionsMap (#1263)
Addendum: tie add and delete of regionMap and encodedRegionsMap

Signed-off-by: stack <stack@apache.org>
2020-03-11 16:57:21 -07:00
stack 0b18a7dc26 HBASE-23929 Shell formatter for for meta table should pretty-print values of info:merge columns 2020-03-11 16:41:30 -07:00
stack 64e67fb98b HBASE-23956 Use less resources running tests (#1266)
Addendum to cleanup pom warning.
2020-03-11 14:35:09 -07:00
Michael Stack aaf9a148a0 HBASE-23956 Use less resources running tests (#1266)
Add being able to configure netty thread counts. Enable socket reuse
(should not have any impact).

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java
 Rename the threads we create in here so they are NOT named same was
 threads created by Hadoop RPC.

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DefaultNettyEventLoopConfig.java
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
 Allow configuring eventloopgroup thread count (so can override for
 tests)

hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
 Enable socket resuse.

hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 Enable socket resuse and config for how many threads to use.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java
 Thread name edit; drop the redundant 'Thread' suffix.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
 Make closeable and shutdown executor when called.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
 Call close on HFileReplicator

hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
 HDFS creates lots of threads. Use less of it so less threads overall.

hbase-server/src/test/resources/hbase-site.xml
hbase-server/src/test/resources/hdfs-site.xml
 Constrain resources when running in test context.

hbase-server/src/test/resources/log4j.properties
 Enable debug on netty to see netty configs in our log

pom.xml
 Add system properties when we launch JVMs to constrain thread counts in
 tests

 Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-11 10:37:50 -07:00
stack d019fd2eed HBASE-23963 Split TestFromClientSide; it takes too long to complete timing out
Split TestGetFromClientSide. Means have to also split the superclass
TestFromClientSideWithCoprocessor.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-10 22:13:24 -07:00
niuyulin e345a95461
HBASE-23953 SimpleBalancer bug when second pass to fill up to min (#1262)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-03-11 10:30:02 +08:00
Nick Dimiduk 9c1f2dd0ca HBASE-23945 Dockerfiles showing hadolint check failures
Signed-off-by: stack <stack@apache.org>
2020-03-10 12:30:56 -07:00
niuyulin 6e6b241e2b
HBASE-23954 SplitParent region should not be balanced (#1264)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-03-10 11:14:43 -07:00
Michael Stack e1e8f396ca HBASE-23851 Log networks and bind addresses when multicast publisher/listener enabled (#1173)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-10 11:00:00 -07:00
Nick Dimiduk 5750876191
HBASE-23932 Minor improvements to Region Normalizer (#1258)
- consolidate checks made by master on behalf of balancer and
   normalizer: deciding if the master is in a healthy state for
   running any actions at all (skipRegionManagementAction). Normalizer
   now does as balancer did previously.
 - both balancer and normalizer make one final check on above
   conditions between calculating an action plan and executing the
   plan. should make the process more responsive to shutdown
   requests.
 - change normalizer to only consider acting on a region when it is in
   the OPEN state. previously we would normalizer attempt to merge a
   region that was already in a MERGING_NEW,MERGING,MERGED state.
 - fix some typos in variable names.

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-03-10 10:08:37 -07:00
Nick Dimiduk 31484f007f
HBASE-23946 [JDK11] Yetus should vote -0 for known JDK11 issues (#1246)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-09 09:10:33 -07:00
Nick Dimiduk 1f77b1ffa5 Revert "HBASE-23945 Dockerfiles showing hadolint check failures"
This reverts commit c61f95022d.
2020-03-07 10:41:54 -08:00
nyl3532016 819965a35a
HBASE-23944 The method setClusterLoad of SimpleLoadBalancer is incorrect when balance by table (#1243)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-03-07 16:18:41 +08:00