Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,505 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

8723 Commits

Author SHA1 Message Date
Andrew Purtell fda324b116
HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions (#3219) 
RegionStates#getAssignmentsForBalancer is used by the HMaster to
collect all regions of interest to the balancer for the next chore
iteration. We check if a table is in disabled state to exclude
regions that will not be of interest (because disabled regions are
or will be offline) or are in a state where they shouldn't be
mutated (like SPLITTING). The current checks are not actually
comprehensive.

Filter out regions not in OPEN or OPENING state when building the
set of interesting regions for the balancer to consider. Only
regions open (or opening) on the cluster are of interest to
balancing calculations for the current iteration. Regions in all
other states can be expected to not be of interest – either offline
(OFFLINE, or FAILED_*), not subject to balancer decisions now
(SPLITTING, SPLITTING_NEW, MERGING, MERGING_NEW), or will be
offline shortly (CLOSING) – until at least the next chore
iteration.

Add TRACE level logging.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-03 18:23:07 -07:00
Andrew Purtell e44592a37d
HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip (#3223) 
We claim in a WARN level log line to be "Playing-it-safe skipping merge/
split gc'ing of regions from hbase:meta while regions-in-transition (RIT)"
but do not actually skip because of a missing return. Remove the warning.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-03 18:14:38 -07:00
Duo Zhang 762abe3bea
HBASE-25838 Use double instead of Double in StochasticLoadBalancer (#3221) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-04 09:04:47 +08:00
Duo Zhang 7640134e3e
HBASE-25774 Added more detailed logs about the restarting of region servers (#3213) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-03 20:33:33 +08:00
GeorryHuang 00fec24c90
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-02 21:30:48 +05:30
Duo Zhang 73a82bd7c6
HBASE-25825 RSGroupBasedLoadBalancer.onConfigurationChange should chain the request to internal balancer (#3209) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-30 22:45:33 +08:00
Duo Zhang 6c65314cdf
HBASE-25819 Fix style issues for StochasticLoadBalancer (#3207) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-29 11:03:55 +08:00
Nick Dimiduk b061b0c4ed HBASE-25779 HRegionServer#compactSplitThread should be private 
Minor refactor. Make the `compactSplitThread` member field of `HRegionServer` private, and gate
all access through the getter method.

Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-04-28 16:46:36 -07:00
Michael Stack 2382f68b23
HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars (#3184) 
Need to add to allowed-licenses list too....

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-04-27 08:37:25 -07:00
Duo Zhang 8856f61986 HBASE-25757 Addendum remove CandidateGenerator classes under hbase-server module 2021-04-27 23:25:51 +08:00
Duo Zhang a4d954e606
HBASE-25757 Move BaseLoadBalancer to hbase-balancer module (#3191) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-26 12:03:25 +08:00
Duo Zhang 7f90c2201f HBASE-25723 Temporarily remove the trace support for RegionScanner.next (#3119) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang f6ff519dd0 HBASE-25591 Upgrade opentelemetry to 0.17.1 (#2971) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang bb8c4967f8 HBASE-25535 Set span kind to CLIENT in AbstractRpcClient (#2907) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 2be2c63f0d HBASE-25484 Add trace support for WAL sync (#2892) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 03e12bfa4a HBASE-25455 Add trace support for HRegion read/write operation (#2861) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang ae2c62ffaa HBASE-25481 Add host and port attribute when tracing rpc call at client side (#2857) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 805b2ae2ad HBASE-23898 Add trace support for simple apis in async client (#2813) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 2420286715 HBASE-25401 Add trace support for async call in rpc client (#2790) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 302d9ea8b8 HBASE-25373 Remove HTrace completely in code base and try to make use of OpenTelemetry 
Signed-off-by: stack <stack@apache.org>
 2021-04-25 09:23:23 +08:00
Andrew Purtell 9895b2dfdf
HBASE-25756 Support alternate compression for major and minor compactions (#3142) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-23 15:45:26 -07:00
Duo Zhang 96fefce9c3
HBASE-25802 Miscellaneous style improvements for load balancer related classes (#3192) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-23 15:20:27 +08:00
haxiaolin 996862c1cc
HBASE-25754 StripeCompactionPolicy should support compacting cold regions (#3152) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-23 14:58:53 +08:00
Toshihiro Suzuki 5f4e2e111b
HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point (#3150) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-04-22 13:53:36 +09:00
Duo Zhang 50920ee306
HBASE-25774 TestSyncReplicationStandbyKillRS#testStandbyKillRegionServer is flaky (#3189) 
Wait for the restarter thread to finish before checking the state
Add more detailed logs

Signed-off-by: meiyi <myimeiyi@gmail.com>
 2021-04-22 10:10:15 +08:00
Duo Zhang d5c5e48839
HBASE-25793 Move BaseLoadBalancer.Cluster to a separated file (#3185) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-22 09:59:49 +08:00
haxiaolin 0d257baf29
HBASE-25763 TestRSGroupsWithACL.setupBeforeClass is flaky (#3158) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-21 14:41:51 +08:00
Duo Zhang 781da1899a
HBASE-25290 Remove table on master related code in balancer implementation (#3162) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-20 21:31:09 +08:00
Nick Dimiduk b65890da1d Revert "HBASE-25739 TableSkewCostFunction need to use aggregated deviation (#3067)" 
This reverts commit 533c84d330.
 2021-04-16 09:35:02 -07:00
Duo Zhang bf78246b4f
HBASE-25775 Use a special balancer to deal with maintenance mode (#3161) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-04-16 09:50:24 +08:00
clarax 533c84d330
HBASE-25739 TableSkewCostFunction need to use aggregated deviation (#3067) 
Signed-off-by: Michael Stack <stack@apache.org>
Reviewed-by: David Manning <david.manning@salesforce.com>
 2021-04-15 13:12:07 -07:00
xiaoyu 6cf4fdde61
HBASE-25776 Use Class.asSubclass to fix the warning in StochasticLoadBalancer.loadCustomCostFunctions (#3163) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-15 23:34:06 +05:30
Nick Dimiduk bc52bca741
HBASE-25770 Http InfoServers should honor gzip encoding when requested (#3159) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2021-04-15 09:07:13 -07:00
Duo Zhang 5910e9e2d1
HBASE-25767 CandidateGenerator.getRandomIterationOrder is too slow on large cluster (#3149) 
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-13 23:00:54 +08:00
Duo Zhang de012d7d1f
HBASE-25759 The master services field in LocalityBasedCostFunction is never used (#3144) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-12 22:27:01 +08:00
Duo Zhang f9e928e5a7
HBASE-25184 Move RegionLocationFinder to hbase-balancer (#2543) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-10 21:10:53 +08:00
Nick Dimiduk 5f1f8be667 HBASE-25744 Change default of `hbase.normalizer.merge.min_region_size.mb` to `0` 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-04-09 15:00:38 -07:00
Geoffrey Jacoby 74e533d5ab
HBASE-25751 - Add writable TimeToPurgeDeletes to ScanOptions (#3137) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-09 13:05:47 -07:00
meiyi ad06aa2082
HBASE-25747 Remove unused getWriteAvailable method in OperationQuota (#3133) 
Signed-off-by: stack <stack@apache.org>
 2021-04-09 10:23:34 +08:00
Pankaj 6444e94c38
HBASE-25717 RegionServer aborted with due to ClassCastException (#3108) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-08 16:27:17 +05:30
stack d9f4f41f76 HBASE-25735 Add target Region to connection exceptions 
Addendum to fix broke compile.
 2021-04-07 07:56:25 -07:00
Jan Hentschel 048ca4e43f
HBASE-25174 Remove deprecated fields in HConstants (#2558) 
Remove the deprecated fields, which can be removed in 3.0.0. Marked the
constant OLDEST_TIMESTAMP as InterfaceAudience.Private as it is only use
in classes, which are also marked as InterfaceAudience.Private.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-03 23:12:16 +08:00
Jan Hentschel 5a63fe65aa
HBASE-25199 Remove deprecated HStore#getStoreHomedir methods (#2562) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-03 23:10:20 +08:00
d-c-manning 7a31557c51
HBASE-25726 MoveCostFunction is not included in the list of cost functions for StochasticLoadBalancer (#3116) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-04-02 09:22:13 -07:00
lujiefsi e14ec57eee
HBASE-25558:Adding audit log for execMasterService (#3101) 
Signed-off-by: stack <stack@apache.org>
 2021-03-31 16:12:31 -07:00
Toshihiro Suzuki 46f7d9dd4b
HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3098) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-03-30 09:18:56 +09:00
Josh Elser 57a49f5ca7
HBASE-25692 Always try to close the WAL reader when we catch any exception (#3090) 
There are code paths in which we throw non-IOExceptions when
initializing a WAL reader. However, we only close the InputStream to the
WAL filesystem when the exception is an IOException. Close it if it is
open in all cases.

Co-authored-by: Josh Elser <jelser@cloudera.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-03-29 12:15:58 -07:00
Michael Stack 6a8998b29c HBASE-25695 Link to the filter on hbase:meta from user tables panel on master page (#3092) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-03-27 20:36:55 -07:00
Toshihiro Suzuki 93b1163a8b
HBASE-25702 Remove RowProcessor (#3097) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-03-28 07:38:42 +09:00
caroliney14 1e4639d2eb
HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2769) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-03-25 10:39:07 -07:00