Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,443 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

71 Commits

Author SHA1 Message Date
Michael Stack 1b5403cf7d
HBASE-27340 Artifacts with resolved profiles (#4740) (#4761) 
(Forward port from branch-2; simplified by the fact that there
is no hadoop-2.0 profile on master branch)

    Make it so our published poms carry the minimum needed to run
    an hbase; the published pom has no profiles -- the profiles
    specified at build time are resolved, their dependencies inlined,
    and then they are stripped -- and no build-time, or plugins
    dependencies or properties, etc. Resultant poms have explicit
    hadoop lib versions baked in -- no more being able to choose
    hbase with hadoop2 or haddop3 at downstream build time by setting
    a '-Dhadoop.profile=X.0'.

    Pattern is to add profiles when none in sub-modules when
    the flatten plugin complains it can't resolve an hadoop
    dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
    Adding the profile in the sub-module make it so the flatten
    plugin can figure 'hadoop.version' definitively.
    (In master there is only the hadoop-3.0 profile).

    Another spin on the above happens when profiles already exist
    in submodule but the flatten plugin is complaining it can't
    figure figure version on an hadoop dependency NOT under
    profiles. Below, we move the delinquent hadoop dependency under
    existing profiles (minikdc was the usual dependency outside
    profiles in sub-modules that flatten complained about).

    Sometimes, moving an hadoop dependency under a profile, there
    would be excludes on the local dependency. If the parent pom
    excludes section was missing the local excludes, we added them
    up to the parent module so all excluding is done up there in
    the parent profile dependencyManagement section.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-09-06 23:06:13 +08:00
Duo Zhang acf144717b HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 10:00:43 +08:00
Andrew Purtell 5af4b3d5c8
HBASE-27202 Clean up error-prone findings in hbase-balancer (#4623) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-18 15:35:43 -07:00
Duo Zhang 9bcb7493e0
HBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master (#4582) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-06-29 11:04:21 +08:00
Duo Zhang 02990894ab
HBASE-27121 Set version as 3.0.0-alpha-3 in master in prep for first RC of 3.0.0-alpha-3 (#4535) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-06-15 10:51:31 +08:00
d-c-manning cec35b6e80
HBASE-27054 TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky (#4454) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-21 09:17:04 -07:00
d-c-manning da55154056
HBASE-26989 TestStochasticLoadBalancer fixes for performance and consistency (#4385) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Reviewed by: Rushabh Shah <shahrs87@gmail.com>
 2022-05-02 10:33:13 -07:00
Duo Zhang 9c8c9e7fbf HBASE-26899 Run spotless:apply 
Closes #4312
 2022-05-01 22:15:09 +08:00
d-c-manning 4a33ed1c61
HBASE-26988 dynamic configuration of loadbalance.bytable (#4384) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-04-29 14:23:32 -07:00
d-c-manning f848c61e3d
HBASE-22349 slop in StochasticLoadBalancer (#4371) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-04-28 12:28:15 -07:00
Duo Zhang 3ae0d9012c
HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless (#4324) 
Signed-off-by: Yulin Niu <niuyulin@apache.org
 2022-04-09 21:38:41 +08:00
Xiaolin Ha ad70ac2cd1
HBASE-26872 Load rate calculator for cost functions should be more precise (#4253) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
 2022-03-27 11:12:00 +08:00
Andrew Purtell 10471944bd
HBASE-26582 Prune use of Random and SecureRandom objects (#4118) 
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.

Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.

Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-08 13:49:02 -08:00
Duo Zhang 19ad3d6654 HBASE-26805 Addendum forward port the test improvements in HBASE-26304 for branch-2 2022-03-07 08:16:21 +08:00
Duo Zhang 480db06e46
HBASE-26805 TestRegionHDFSBlockLocationFinder is flaky (#4169) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-03-07 08:08:04 +08:00
Duo Zhang 443bf30a5d
HBASE-26643 LoadBalancer should not return empty map (#4005) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2022-01-07 23:50:37 +08:00
Duo Zhang 3f59f21be0
HBASE-26621 Set version as 3.0.0-alpha-3-SNAPSHOT in master (#3978) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-12-24 14:20:32 +08:00
Duo Zhang 8bca21b47d
HBASE-26558 Set version as 3.0.0-alpha-2 in master in prep for first RC of 3.0.0-alpha-2 (#3935) 
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
 2021-12-11 20:52:35 +08:00
Bryan Beaudreault 1b27124c61
HBASE-26304 Reflect out of band locality improvements in metrics and balancer (#3803) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-27 12:01:46 +08:00
Duo Zhang 33287ac502
HBASE-26455 TestStochasticLoadBalancerRegionReplicaWithRacks fails consistently (#3879) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2021-11-25 21:40:46 +08:00
Andrew Purtell 69a4eda35f
HBASE-26443 Some BaseLoadBalancer log lines should be at DEBUG level (#3838) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-11-15 16:05:27 -08:00
clarax 62cd2b688e
HBASE-26337 Optimization for weighted random generators (#3732) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-09 08:17:21 +08:00
clarax cd7a5099d4
HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3724) 
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-11-03 11:57:37 -07:00
clarax 72a88468a8
HBASE-26309 Balancer tends to move regions to the server at the end of list (#3723) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-02 21:56:38 +08:00
clarax 7af5277859
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering Bal… (#3729) 
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-10-25 11:29:31 -07:00
clarax 0d4982404c
HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger (#3710) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-10-21 11:19:21 -07:00
GeorryHuang 63306942b1
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run (#3678) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-09-28 21:20:48 +08:00
clarax cbebf85b3c
HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison (#3698) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-26 15:10:44 +08:00
clarax 2b26dfbaf4
HBASE-26178 Improve data structure and algorithm for BalanceClusterState to improve computation speed for large cluster (#3682) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-25 20:02:28 +08:00
clarax afbdd4115b
HBASE-26237 Improve computation complexity for primaryRegionCountSkewCostFunction (#3657) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-09-03 18:36:55 -07:00
Duo Zhang 16721239e7
HBASE-26100 Set version as 3.0.0-alpha-2-SNAPSHOT in master (#3508) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-20 23:04:08 +08:00
Duo Zhang d30cc27097
HBASE-26081 Copy HBTU to hbase-testing-util, rename the HBTU related classes in hbase-server and mark them as IA.LimitedPrivate (#3478) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-19 09:29:08 +08:00
clarax 68aaf1ff02
HBASE-25739 TableSkewCostFunction need to use aggregated deviation (#3415) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-07-13 08:24:11 -07:00
clarax 1e763d521f
HBASE-25973 Balancer should explain progress in a better way in log (#3356) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2021-07-12 10:10:24 -07:00
Duo Zhang 5118321ec9
HBASE-26059 Set version as 3.0.0-alpha-1 in master in prep for first RC of 3.0.0-alpha-1 (#3453) 
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-07-02 07:50:41 +08:00
Duo Zhang 480b6bb637
HBASE-25995 Change the method name for DoubleArrayCost.setCosts (#3381) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-15 11:53:00 +08:00
Andrew Purtell 335305e0cf
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302) 
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-01 09:57:48 -07:00
Duo Zhang 06c6e06803
HBASE-25916 Move FavoredNodeLoadBalancer to hbase-balancer module (#3327) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-31 22:55:04 +08:00
Duo Zhang f2ff816532
HBASE-25939 Move more tests code for StochasticLoadBalancer to hbase-balancer module (#3331) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-30 22:00:18 +08:00
Duo Zhang 7218c83f81
HBASE-25931 Move FavoredNodeManager to hbase-balancer module (#3324) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-28 15:57:59 +08:00
Duo Zhang ed8df5eded
HBASE-25758 Move MetaTableAccessor out of hbase-balancer module (#3309) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-28 09:19:07 +08:00
Duo Zhang 63141bf576
HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code (#3313) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-27 16:05:14 +08:00
Duo Zhang 76fbb8b965
HBASE-25818 Move StochasticLoadBalancer to hbase-balancer module (#3206) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-25 23:24:35 +08:00
Duo Zhang f94f4e29fe
HBASE-25873 Refactor and cleanup the code for CostFunction (#3274) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-24 18:14:55 +08:00
Duo Zhang 1c6994ad70
HBASE-25872 Add documentation for LoadBalancer about synchronization (#3267) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-05-17 10:17:24 +08:00
Duo Zhang 85d8ec7dae
HBASE-25883 The regionFinder and rackManager fields in BaseLoadBalancer should be volatile (#3262) 
Signed-off-by: meiyi <myimeiyi@gmail.com>
 2021-05-15 20:20:03 +08:00
Duo Zhang 29bd3dd586
HBASE-25852 Move all the intialization work of LoadBalancer implementation to initialize method (#3248) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-11 22:03:33 +08:00
Duo Zhang ba4cb91211
HBASE-25851 Make LoadBalancer not extend Configurable interface (#3233) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-06 16:11:46 +08:00
Duo Zhang 90f986497b
HBASE-25834 Remove balanceTable method from LoadBalancer interface (#3217) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-05 15:48:01 +08:00
GeorryHuang 00fec24c90
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-02 21:30:48 +05:30