Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,493 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

212 Commits

Author SHA1 Message Date
Yutong Xiao 713ed46d03 HBASE-27246 RSGroupMappingScript#getRSGroup has thread safety problem (#4657) 
Co-authored-by: Sean Xiao Yutong <sean.xiaoyt@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b44bfc52cc)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
 2022-08-24 15:38:26 +08:00
Duo Zhang db98fab114 HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky (#4636) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 31fc97edee)

Conflicts:
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestUpdateRSGroupConfiguration.java
 2022-07-21 14:52:39 +08:00
Duo Zhang 99f2ab5aa8 HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 10:00:31 +08:00
Andrew Purtell bcf3cd55fd Amend HBASE-27082 spotless:apply 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-01 10:50:44 -07:00
Duo Zhang 716a56c849 HBASE-27082 Change the return value of RSGroupInfo.getServers from SortedSet to Set to keep compatibility (#4480) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestUpdateRSGroupConfiguration.java
 2022-06-01 09:17:34 -07:00
Andrew Purtell 6dd1a6bef0 HBASE-27081 Fix disallowed compatibility breaks on branch-2.5 and branch-2 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-31 19:56:16 -07:00
Duo Zhang 1a5b1b266c HBASE-26899 Run spotless:apply 2022-05-01 22:41:49 +08:00
Andrew Purtell 300f9b9576 HBASE-26582 Prune use of Random and SecureRandom objects (#4118) 
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.

Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.

Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-08 15:22:00 -08:00
GeorryHuang 08800e19d6
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run (#3706) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-09-29 21:25:39 +08:00
Bryan Beaudreault e07928df4c HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Re-application of HBASE-26147 with the correct author metadata)
 2021-09-01 21:58:30 -04:00
Josh Elser 1e74f08403 Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed" 
This reverts commit 5b58d11c91.

I accidentally applied this as myself instead of as Bryan. Reverting it
to apply the patch properly.
 2021-09-01 21:57:46 -04:00
Josh Elser 5b58d11c91 HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
 2021-09-01 21:47:53 -04:00
Andrew Purtell a4e8ee183e HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302) 
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
	hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 2021-06-01 12:41:15 -07:00
Baiqiang Zhao cd06870f07
HBASE-25682 Add a new command to update the configuration of all RSs in a RSGroup for branch-2 (#3106) 
* HBASE-25682 Add a new command to update the configuration of all RSs in a RSGroup

Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-05-21 22:49:48 +05:30
Duo Zhang 8110b18ab0 HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-18 10:43:01 +08:00
Duo Zhang 72d7641fb5
HBASE-25852 Move all the intialization work of LoadBalancer implementation to initialize method (#3248) (#3258) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-13 14:19:03 +08:00
Duo Zhang 2d4904ceea
HBASE-25851 Make LoadBalancer not extend Configurable interface (#3233) (#3235) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-06 23:39:43 +08:00
Duo Zhang afa9836b87 HBASE-25834 Remove balanceTable method from LoadBalancer interface (#3217) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-05 16:03:51 +08:00
Duo Zhang 3ebcfda3c7 HBASE-25825 RSGroupBasedLoadBalancer.onConfigurationChange should chain the request to internal balancer (#3209) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-30 22:57:18 +08:00
haxiaolin 4b916ea0eb HBASE-25763 TestRSGroupsWithACL.setupBeforeClass is flaky (#3158) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-21 14:46:12 +08:00
Mohammad Arshad 0487760829
HBASE-25492: Create table with RSGroup (#2883) 
Added new interface in TableDescriptor which allows user to define RSGroup name while creating or modifying a table.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-03-06 10:44:19 +05:30
mokai 2b25fca6d8
HBASE-24755 [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup (#2847) (#2846) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-01-06 19:58:47 +05:30
haxl 32c4432b9b
HBASE-25334 TestRSGroupsFallback.testFallback is flaky (#2775) 
Signed-off-by: stack <stack@apache.org>
 2020-12-16 13:21:53 -08:00
Baiqiang Zhao b6c03f2027
HBASE-25365 The log in move_servers_rsgroup is incorrect (#2743) 
Signed-off-by: stack <stack@apache.org>
 2020-12-16 09:13:57 -08:00
lujiefsi a1db679e4d
HBASE-25332:fix One pontential NPE 
Closes #2715

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-01 19:47:09 +05:30
Baiqiang Zhao e927fe88fb
HBASE-25330 RSGroupInfoManagerImpl#moveServers return is not set of servers moved (#2719) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-11-30 11:40:51 +08:00
Andrew Purtell 1b6399a480
HBASE-24640 [branch-2] Purge use of VisibleForTesting (#2696) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-11-25 14:15:32 -08:00
Baiqiang Zhao fdff8ef46d HBASE-25298 hbase.rsgroup.fallback.enable should support dynamic configuration (#2668) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-11-19 09:20:43 +08:00
niuyulin b0d140f40a
HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE (#2534) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-10-13 21:25:09 +08:00
Mohammad Arshad 74e111633e
HBASE-24025:Improve performance of move_servers_rsgroup and move_tables_rsgroup by using async region move API (#1321) 2020-10-09 12:12:09 -07:00
Duo Zhang fe937242c1 HBASE-25080 Should not use AssignmentManager to test whether a table is enabled or not (#2436) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-10-03 21:16:47 +08:00
Mohammad Arshad aac21aebec
HBASE-24719: Renaming invalid rsgroup throws NPE instead of proper error message (#2074) 
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2020-09-04 23:04:32 +05:30
XinSun c7b930a9e8
HBASE-24759 Refuse to update configuration of default group (#2350) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-09-04 18:45:12 +08:00
XinSun ad177059cf
HBASE-24760 Add a config hbase.rsgroup.fallback.enable for RSGroup fallback feature (#2325) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-08-31 10:53:05 +08:00
niuyulin a76773a402
HBASE-24928 balanceRSGroup should skip generating balance plan for disabled table and splitParent region (#2300) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-08-26 14:01:35 +08:00
Guanghao Zhang 4667a971b1 HBASE-24912 Enlarge MemstoreFlusherChore/CompactionChecker period for unit test (#2285) 
Signed-off-by: stack <stack@apache.org>
 2020-08-21 12:42:11 +08:00
Mohammad Arshad e08da3bf26
HBASE-24675: On Master restart all servers are assigned to default rsgroup 
Closes #2053

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-17 22:46:43 +05:30
Mohammad Arshad ad826a43a2
HBASE-24721: rename_rsgroup overwriting the existing rsgroup 
Closes #2066

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-15 18:30:16 +05:30
XinSun 3a33f642bf
HBASE-22738 Fallback to default group to choose RS when there are no RS in current group (#2039) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-07-10 09:59:58 +08:00
XinSun 5fcffae5db
HBASE-24431 RSGroupInfo add configuration map to store something extra (#2031) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-07-08 14:39:41 +08:00
XinSun 69d8d4b894 HBASE-24399 [Flakey Tests] Some UTs about RSGroup should wait RSGroupInfoManager to be online (#1738) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-05-20 18:04:33 +08:00
XinSun 80013ec11d HBASE-24135 TableStateNotFoundException happends when table creation if rsgroup is enable (#1550) 
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-05-17 20:50:04 +08:00
Duo Zhang dc2146069c
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for … (#1697) 
Signed-off-by: stack <stack@apache.org>
 2020-05-13 17:59:21 +08:00
Reid Chan 11ef0fd752
HBASE-24345 [ACL] renameRSGroup should require Admin level permission (#1686) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
 2020-05-09 23:33:27 +08:00
Gkkkk302 22912ff346
HBASE-24151 The master server aborted for IllegalThreadStateException (#1507) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-04-16 15:41:26 +05:30
Reid Chan 4c83067e1e
HBASE-24112 [RSGroup] Support renaming rsgroup (#1520) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-04-15 19:04:40 +08:00
niuyulin 244b308a3e
HBASE-23949 refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced (#1324) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-03-25 11:27:32 +08:00
Reid Chan 304edcf1af HBASE-24032 [RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations 2020-03-24 20:12:00 +08:00
stack 50161f2de4 HBASE-24034 [Flakey Tests] A couple of fixes and cleanups 
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupMajorCompactionTTL.java
 Remove spurious assert. Just before this it waits an arbitrary 10
 seconds. Compactions could have completed inside this time. The spirit
 of the test remains.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
 Get log cleaner to go down promptly; its sticking around. See if this
 helps with TestMasterShutdown

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
 We get a rare NPE trying to sync. Make local copy of SyncFuture and see
 if that helps.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
 Compaction  may have completed when not expected; allow for it.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java
 Add wait before testing. Compaction may not have completed. Let
 compaction complete before progressing and then test for empty cache.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestDefaultLoadBalancer.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
 Wait till online before we try and do compaction (else request is
 ignored)

hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 Disable test that fails randomly w/ mockito complaint on some mac os
 x's.

TestMasterShutdown... fix NPE in RSRpcDispatcher... catch it and covert
to false and have master check for successful startup.
 2020-03-23 16:21:19 -07:00
Mohammad Arshad e6adb79741
HBASE-24019: Correct exception messages for table null and namespace unavailable. (#1312) 
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-03-21 20:34:55 +05:30