63cdd026f0
HBASE-27401 Clean up current broken 'n's in our javadoc ( #4812 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-10-06 18:17:34 +08:00
1b5403cf7d
HBASE-27340 Artifacts with resolved profiles ( #4740 ) ( #4761 )
...
(Forward port from branch-2; simplified by the fact that there
is no hadoop-2.0 profile on master branch)
Make it so our published poms carry the minimum needed to run
an hbase; the published pom has no profiles -- the profiles
specified at build time are resolved, their dependencies inlined,
and then they are stripped -- and no build-time, or plugins
dependencies or properties, etc. Resultant poms have explicit
hadoop lib versions baked in -- no more being able to choose
hbase with hadoop2 or haddop3 at downstream build time by setting
a '-Dhadoop.profile=X.0'.
Pattern is to add profiles when none in sub-modules when
the flatten plugin complains it can't resolve an hadoop
dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
Adding the profile in the sub-module make it so the flatten
plugin can figure 'hadoop.version' definitively.
(In master there is only the hadoop-3.0 profile).
Another spin on the above happens when profiles already exist
in submodule but the flatten plugin is complaining it can't
figure figure version on an hadoop dependency NOT under
profiles. Below, we move the delinquent hadoop dependency under
existing profiles (minikdc was the usual dependency outside
profiles in sub-modules that flatten complained about).
Sometimes, moving an hadoop dependency under a profile, there
would be excludes on the local dependency. If the parent pom
excludes section was missing the local excludes, we added them
up to the parent module so all excluding is done up there in
the parent profile dependencyManagement section.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-06 23:06:13 +08:00
acf144717b
HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 10:00:43 +08:00
5af4b3d5c8
HBASE-27202 Clean up error-prone findings in hbase-balancer ( #4623 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-07-18 15:35:43 -07:00
9bcb7493e0
HBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master ( #4582 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2022-06-29 11:04:21 +08:00
02990894ab
HBASE-27121 Set version as 3.0.0-alpha-3 in master in prep for first RC of 3.0.0-alpha-3 ( #4535 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2022-06-15 10:51:31 +08:00
cec35b6e80
HBASE-27054 TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky ( #4454 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-21 09:17:04 -07:00
da55154056
HBASE-26989 TestStochasticLoadBalancer fixes for performance and consistency ( #4385 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Reviewed by: Rushabh Shah <shahrs87@gmail.com>
2022-05-02 10:33:13 -07:00
9c8c9e7fbf
HBASE-26899 Run spotless:apply
...
Closes #4312
2022-05-01 22:15:09 +08:00
4a33ed1c61
HBASE-26988 dynamic configuration of loadbalance.bytable ( #4384 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-04-29 14:23:32 -07:00
f848c61e3d
HBASE-22349 slop in StochasticLoadBalancer ( #4371 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-04-28 12:28:15 -07:00
3ae0d9012c
HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless ( #4324 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org
2022-04-09 21:38:41 +08:00
ad70ac2cd1
HBASE-26872 Load rate calculator for cost functions should be more precise ( #4253 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2022-03-27 11:12:00 +08:00
10471944bd
HBASE-26582 Prune use of Random and SecureRandom objects ( #4118 )
...
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.
Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.
Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-08 13:49:02 -08:00
19ad3d6654
HBASE-26805 Addendum forward port the test improvements in HBASE-26304 for branch-2
2022-03-07 08:16:21 +08:00
480db06e46
HBASE-26805 TestRegionHDFSBlockLocationFinder is flaky ( #4169 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2022-03-07 08:08:04 +08:00
443bf30a5d
HBASE-26643 LoadBalancer should not return empty map ( #4005 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2022-01-07 23:50:37 +08:00
3f59f21be0
HBASE-26621 Set version as 3.0.0-alpha-3-SNAPSHOT in master ( #3978 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-24 14:20:32 +08:00
8bca21b47d
HBASE-26558 Set version as 3.0.0-alpha-2 in master in prep for first RC of 3.0.0-alpha-2 ( #3935 )
...
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
2021-12-11 20:52:35 +08:00
1b27124c61
HBASE-26304 Reflect out of band locality improvements in metrics and balancer ( #3803 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-27 12:01:46 +08:00
33287ac502
HBASE-26455 TestStochasticLoadBalancerRegionReplicaWithRacks fails consistently ( #3879 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2021-11-25 21:40:46 +08:00
69a4eda35f
HBASE-26443 Some BaseLoadBalancer log lines should be at DEBUG level ( #3838 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-11-15 16:05:27 -08:00
62cd2b688e
HBASE-26337 Optimization for weighted random generators ( #3732 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-09 08:17:21 +08:00
cd7a5099d4
HBASE-26311 Balancer gets stuck in cohosted replica distribution ( #3724 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-11-03 11:57:37 -07:00
72a88468a8
HBASE-26309 Balancer tends to move regions to the server at the end of list ( #3723 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-02 21:56:38 +08:00
7af5277859
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering Bal… ( #3729 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-10-25 11:29:31 -07:00
0d4982404c
HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger ( #3710 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-10-21 11:19:21 -07:00
63306942b1
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run ( #3678 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-09-28 21:20:48 +08:00
cbebf85b3c
HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison ( #3698 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-26 15:10:44 +08:00
2b26dfbaf4
HBASE-26178 Improve data structure and algorithm for BalanceClusterState to improve computation speed for large cluster ( #3682 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-25 20:02:28 +08:00
afbdd4115b
HBASE-26237 Improve computation complexity for primaryRegionCountSkewCostFunction ( #3657 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-09-03 18:36:55 -07:00
16721239e7
HBASE-26100 Set version as 3.0.0-alpha-2-SNAPSHOT in master ( #3508 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-20 23:04:08 +08:00
d30cc27097
HBASE-26081 Copy HBTU to hbase-testing-util, rename the HBTU related classes in hbase-server and mark them as IA.LimitedPrivate ( #3478 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-19 09:29:08 +08:00
68aaf1ff02
HBASE-25739 TableSkewCostFunction need to use aggregated deviation ( #3415 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-07-13 08:24:11 -07:00
1e763d521f
HBASE-25973 Balancer should explain progress in a better way in log ( #3356 )
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2021-07-12 10:10:24 -07:00
5118321ec9
HBASE-26059 Set version as 3.0.0-alpha-1 in master in prep for first RC of 3.0.0-alpha-1 ( #3453 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-07-02 07:50:41 +08:00
480b6bb637
HBASE-25995 Change the method name for DoubleArrayCost.setCosts ( #3381 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-15 11:53:00 +08:00
335305e0cf
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime ( #3302 )
...
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-06-01 09:57:48 -07:00
06c6e06803
HBASE-25916 Move FavoredNodeLoadBalancer to hbase-balancer module ( #3327 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-31 22:55:04 +08:00
f2ff816532
HBASE-25939 Move more tests code for StochasticLoadBalancer to hbase-balancer module ( #3331 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-30 22:00:18 +08:00
7218c83f81
HBASE-25931 Move FavoredNodeManager to hbase-balancer module ( #3324 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-28 15:57:59 +08:00
ed8df5eded
HBASE-25758 Move MetaTableAccessor out of hbase-balancer module ( #3309 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-28 09:19:07 +08:00
63141bf576
HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code ( #3313 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-27 16:05:14 +08:00
76fbb8b965
HBASE-25818 Move StochasticLoadBalancer to hbase-balancer module ( #3206 )
...
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
2021-05-25 23:24:35 +08:00
f94f4e29fe
HBASE-25873 Refactor and cleanup the code for CostFunction ( #3274 )
...
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
2021-05-24 18:14:55 +08:00
1c6994ad70
HBASE-25872 Add documentation for LoadBalancer about synchronization ( #3267 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-05-17 10:17:24 +08:00
85d8ec7dae
HBASE-25883 The regionFinder and rackManager fields in BaseLoadBalancer should be volatile ( #3262 )
...
Signed-off-by: meiyi <myimeiyi@gmail.com>
2021-05-15 20:20:03 +08:00
29bd3dd586
HBASE-25852 Move all the intialization work of LoadBalancer implementation to initialize method ( #3248 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-11 22:03:33 +08:00
ba4cb91211
HBASE-25851 Make LoadBalancer not extend Configurable interface ( #3233 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-06 16:11:46 +08:00
90f986497b
HBASE-25834 Remove balanceTable method from LoadBalancer interface ( #3217 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-05 15:48:01 +08:00
Duo Zhang
Michael Stack
Andrew Purtell
d-c-manning
Xiaolin Ha
Bryan Beaudreault
clarax
GeorryHuang