Commit Graph

199 Commits

Author SHA1 Message Date
Duo Zhang 02dd256703 HBASE-26899 Run spotless:apply 2022-05-01 23:10:46 +08:00
Andrew Purtell 210517040b HBASE-26582 Prune use of Random and SecureRandom objects (#4118)
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.

Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.

Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-08 16:12:08 -08:00
Andrew Purtell 2fec951303 Revert "HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268)"
This reverts commit 2bf86468d8.
2021-05-19 12:29:12 -07:00
Andrew Purtell ca3b267130 Revert "Amend HBASE-25032: Compliation fix for TestRSGroupsKillRS"
This reverts commit 4d44bfa8ad.
2021-05-19 12:29:03 -07:00
Andrew Purtell 4d44bfa8ad Amend HBASE-25032: Compliation fix for TestRSGroupsKillRS 2021-05-19 10:54:33 -07:00
Duo Zhang 2bf86468d8 HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-18 10:43:13 +08:00
Duo Zhang a7814bbe57 HBASE-25825 RSGroupBasedLoadBalancer.onConfigurationChange should chain the request to internal balancer (#3209)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-30 22:57:29 +08:00
Mohammad Arshad 9ec86d97a3
HBASE-25492: Create table with RSGroup (#3033)
Added a new interface in TableDescriptor which allows user to define RSGroup name while creating or modifying a table.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2021-03-08 23:11:15 +05:30
mokai 0a4685b085
HBASE-24755 [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup (#2847) (#2846)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-01-06 20:00:45 +05:30
haxl 98b7aa7765 HBASE-25334 TestRSGroupsFallback.testFallback is flaky (#2775)
Signed-off-by: stack <stack@apache.org>
2020-12-16 13:23:50 -08:00
Baiqiang Zhao 722cbe4294 HBASE-25365 The log in move_servers_rsgroup is incorrect (#2743)
Signed-off-by: stack <stack@apache.org>
2020-12-16 09:15:50 -08:00
lujiefsi a1db679e4d
HBASE-25332:fix One pontential NPE
Closes #2715

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-01 19:47:09 +05:30
Baiqiang Zhao e927fe88fb
HBASE-25330 RSGroupInfoManagerImpl#moveServers return is not set of servers moved (#2719)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-30 11:40:51 +08:00
Andrew Purtell 1b6399a480
HBASE-24640 [branch-2] Purge use of VisibleForTesting (#2696)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-11-25 14:15:32 -08:00
Baiqiang Zhao fdff8ef46d HBASE-25298 hbase.rsgroup.fallback.enable should support dynamic configuration (#2668)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-19 09:20:43 +08:00
niuyulin b0d140f40a
HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE (#2534)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-13 21:25:09 +08:00
Mohammad Arshad 74e111633e
HBASE-24025:Improve performance of move_servers_rsgroup and move_tables_rsgroup by using async region move API (#1321) 2020-10-09 12:12:09 -07:00
Duo Zhang fe937242c1 HBASE-25080 Should not use AssignmentManager to test whether a table is enabled or not (#2436)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-03 21:16:47 +08:00
Mohammad Arshad aac21aebec
HBASE-24719: Renaming invalid rsgroup throws NPE instead of proper error message (#2074)
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2020-09-04 23:04:32 +05:30
XinSun c7b930a9e8
HBASE-24759 Refuse to update configuration of default group (#2350)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-04 18:45:12 +08:00
XinSun ad177059cf
HBASE-24760 Add a config hbase.rsgroup.fallback.enable for RSGroup fallback feature (#2325)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-31 10:53:05 +08:00
niuyulin a76773a402
HBASE-24928 balanceRSGroup should skip generating balance plan for disabled table and splitParent region (#2300)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-26 14:01:35 +08:00
Guanghao Zhang 4667a971b1 HBASE-24912 Enlarge MemstoreFlusherChore/CompactionChecker period for unit test (#2285)
Signed-off-by: stack <stack@apache.org>
2020-08-21 12:42:11 +08:00
Mohammad Arshad e08da3bf26
HBASE-24675: On Master restart all servers are assigned to default rsgroup
Closes #2053

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-17 22:46:43 +05:30
Mohammad Arshad ad826a43a2
HBASE-24721: rename_rsgroup overwriting the existing rsgroup
Closes #2066

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-15 18:30:16 +05:30
XinSun 3a33f642bf
HBASE-22738 Fallback to default group to choose RS when there are no RS in current group (#2039)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 09:59:58 +08:00
XinSun 5fcffae5db
HBASE-24431 RSGroupInfo add configuration map to store something extra (#2031)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-08 14:39:41 +08:00
XinSun 69d8d4b894 HBASE-24399 [Flakey Tests] Some UTs about RSGroup should wait RSGroupInfoManager to be online (#1738)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-05-20 18:04:33 +08:00
XinSun 80013ec11d HBASE-24135 TableStateNotFoundException happends when table creation if rsgroup is enable (#1550)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-17 20:50:04 +08:00
Duo Zhang dc2146069c
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for … (#1697)
Signed-off-by: stack <stack@apache.org>
2020-05-13 17:59:21 +08:00
Reid Chan 11ef0fd752
HBASE-24345 [ACL] renameRSGroup should require Admin level permission (#1686)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2020-05-09 23:33:27 +08:00
Gkkkk302 22912ff346
HBASE-24151 The master server aborted for IllegalThreadStateException (#1507)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-16 15:41:26 +05:30
Reid Chan 4c83067e1e
HBASE-24112 [RSGroup] Support renaming rsgroup (#1520)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 19:04:40 +08:00
niuyulin 244b308a3e
HBASE-23949 refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced (#1324)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-03-25 11:27:32 +08:00
Reid Chan 304edcf1af HBASE-24032 [RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations 2020-03-24 20:12:00 +08:00
stack 50161f2de4 HBASE-24034 [Flakey Tests] A couple of fixes and cleanups
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupMajorCompactionTTL.java
 Remove spurious assert. Just before this it waits an arbitrary 10
 seconds. Compactions could have completed inside this time. The spirit
 of the test remains.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
 Get log cleaner to go down promptly; its sticking around. See if this
 helps with TestMasterShutdown

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
 We get a rare NPE trying to sync. Make local copy of SyncFuture and see
 if that helps.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
 Compaction  may have completed when not expected; allow for it.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java
 Add wait before testing. Compaction may not have completed. Let
 compaction complete before progressing and then test for empty cache.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestDefaultLoadBalancer.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
 Wait till online before we try and do compaction (else request is
 ignored)

hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 Disable test that fails randomly w/ mockito complaint on some mac os
 x's.

TestMasterShutdown... fix NPE in RSRpcDispatcher... catch it and covert
to false and have master check for successful startup.
2020-03-23 16:21:19 -07:00
Mohammad Arshad e6adb79741
HBASE-24019: Correct exception messages for table null and namespace unavailable. (#1312)
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-21 20:34:55 +05:30
wen_yi 292cadf847 HBASE-23971 protoc warns: "no protobuf syntax specified" 2020-03-17 15:38:50 -07:00
Michael Stack 2655f9647e
HBASE-23956 Use less resources running tests (#1266)
Add being able to configure netty thread counts. Enable socket reuse
(should not have any impact).

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java
 Rename the threads we create in here so they are NOT named same was
 threads created by Hadoop RPC.

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DefaultNettyEventLoopConfig.java
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
 Allow configuring eventloopgroup thread count (so can override for
 tests)

hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
 Enable socket resuse.

hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 Enable socket resuse and config for how many threads to use.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java
 Thread name edit; drop the redundant 'Thread' suffix.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
 Make closeable and shutdown executor when called.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
 Call close on HFileReplicator

hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
 HDFS creates lots of threads. Use less of it so less threads overall.

hbase-server/src/test/resources/hbase-site.xml
hbase-server/src/test/resources/hdfs-site.xml
 Constrain resources when running in test context.

hbase-server/src/test/resources/log4j.properties
 Enable debug on netty to see netty configs in our log

pom.xml
 Add system properties when we launch JVMs to constrain thread counts in
 tests

 Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-11 10:25:11 -07:00
Duo Zhang ba34d95b49 HBASE-23809 The RSGroup shell test is missing (#1142)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-02-08 22:51:03 +08:00
stack f7eeea0789 HBASE-23729 [Flakeys] TestRSGroupsBasics#testClearNotProcessedDeadServer fails most of the time 2020-01-23 16:44:49 -08:00
Guanghao Zhang bd5b0ba55d HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish (#996)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-09 18:39:20 +08:00
haxiaolin fff0f33c5a HBASE-22767 System table RIT STUCK if their RSGroup has no highest version RSes
Signed-off-by: stack <stack@apache.org>
2019-10-08 16:32:07 -07:00
Duo Zhang 45cbe8b038 HBASE-23125 TestRSGroupsAdmin2 is flaky 2019-10-07 18:46:57 +08:00
Duo Zhang 2c118c6c01 HBASE-23006 RSGroupBasedLoadBalancer should also try to place replicas for the same region to different region servers (#605)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-11 11:38:23 +08:00
linkaline fae0e18a73 HBASE-22642 Make move operations of RSGroup idempotent (#387)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-02 17:22:38 +08:00
Duo Zhang 4a4208f872 HBASE-22933 Do not need to kick reassign for rs group change any more (#550)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-30 11:07:02 +08:00
Jan Hentschel 2338269905
HBASE-22764 Fixed remaining Checkstyle issues in the hbase-rsgroup module
Signed-off-by: stack <stack@apache.org>
2019-07-30 08:57:10 +02:00
syedmurtazahassan cab92111e4 HBASE-22702 [Log] 'Group not found for table' is chatty
Signed-off-by Reid Chan <reidchan@apache.org>
2019-07-25 23:16:30 +08:00
linkaline abf3d183cd HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM's pendingAssginQueue is cleared (#354)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-17 22:11:24 +08:00