Commit Graph

8361 Commits

Author SHA1 Message Date
Duo Zhang c52c091609 HBASE-25819 Fix style issues for StochasticLoadBalancer (#3207)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-29 11:26:58 +08:00
Nick Dimiduk 3a01941963 HBASE-25779 HRegionServer#compactSplitThread should be private
Minor refactor. Make the `compactSplitThread` member field of `HRegionServer` private, and gate
all access through the getter method.

Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-04-28 16:54:44 -07:00
stack 04c30bda88 HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars (#3184)
Need to add to allowed-licenses list too....

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-04-27 08:41:15 -07:00
Andrew Purtell 01ce44abc4 HBASE-25756 Support alternate compression for major and minor compactions (#3142)
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
	hbase-shell/src/main/ruby/hbase/admin.rb
2021-04-23 16:02:41 -07:00
Duo Zhang f374357cc3 HBASE-25802 Miscellaneous style improvements for load balancer related classes (#3192)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-23 16:32:10 +08:00
haxiaolin fd7b2c6eec HBASE-25754 StripeCompactionPolicy should support compacting cold regions (#3152)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-23 15:31:55 +08:00
Duo Zhang a02ce95ff1
HBASE-25793 Move BaseLoadBalancer.Cluster to a separated file (#3185) (#3190)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-23 00:08:16 +08:00
Toshihiro Suzuki f4f84302fa HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-04-22 14:06:28 +09:00
Duo Zhang a177fb4c62 HBASE-25775 Use a special balancer to deal with maintenance mode (#3161)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-04-16 13:39:48 +08:00
Nick Dimiduk 32207cd478 HBASE-25770 Http InfoServers should honor gzip encoding when requested (#3159)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-04-15 13:46:53 -07:00
xiaoyu 819062bac3
HBASE-25776 Use Class.asSubclass to fix the warning in StochasticLoadBalancer.loadCustomCostFunctions (#3163)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-15 23:35:33 +05:30
DivyeshChandra 35520a72fb HBASE-25653 Add units and round off region size to 2 digits after decimal (#3046)
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-04-13 09:06:05 -07:00
Duo Zhang 15659f0e85 HBASE-25767 CandidateGenerator.getRandomIterationOrder is too slow on large cluster (#3149)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-13 23:14:48 +08:00
Baiqiang Zhao ae3daa5c0b HBASE-25482 Improve SimpleRegionNormalizer#getAverageRegionSizeMb (#2858)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-04-12 15:19:43 -07:00
Duo Zhang 702220a135 HBASE-25759 The master services field in LocalityBasedCostFunction is never used (#3144)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-12 22:31:05 +08:00
Nick Dimiduk b071902b10 HBASE-25744 Change default of `hbase.normalizer.merge.min_region_size.mb` to `0`
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-04-11 16:22:02 -07:00
Geoffrey Jacoby 89c9f28d3f HBASE-25751 - Add writable TimeToPurgeDeletes to ScanOptions (#3137)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-09 13:09:24 -07:00
meiyi 3faa67520c HBASE-25747 Remove unused getWriteAvailable method in OperationQuota (#3133)
Signed-off-by: stack <stack@apache.org>
2021-04-09 10:41:51 +08:00
Pankaj 89fa8e7333
HBASE-25717 RegionServer aborted with due to ClassCastException (#3108)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-08 16:28:41 +05:30
Baiqiang Zhao 8ff17c68e2
HBASE-25687 Backport "HBASE-25681 Add a switch for server/table query… (#3074)
Signed-off-by: stack <stack@apache.org>
2021-04-07 11:11:46 -07:00
Jan Hentschel 18882d6f9a
HBASE-24305 Prepare deprecations in ServerName (#1666) (#3121)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-04-05 22:10:04 +02:00
Jan Hentschel 1fa07cea80 HBASE-25199 Remove deprecated HStore#getStoreHomedir methods (#2562)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-03 23:44:23 +08:00
d-c-manning 554e039817 HBASE-25726 MoveCostFunction is not included in the list of cost functions for StochasticLoadBalancer (#3116)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-04-02 10:43:51 -07:00
stack b051ad5594 Revert "HBASE-25713 Make an hbase-wal module"
This reverts commit e58f1b24a6.
2021-03-31 16:15:50 -07:00
lujiefsi d90d0c2231 HBASE-25558:Adding audit log for execMasterService (#3101)
Signed-off-by: stack <stack@apache.org>
2021-03-31 16:14:50 -07:00
stack e58f1b24a6 HBASE-25713 Make an hbase-wal module
First extract an hbase-coprocessor module used by hbase-client, hbase-server.
This is prerequisite to extracting an hbase-wal module.

M hbase-common/src/main/java/org/apache/hadoop/hbase/Abortable.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/DoNotRetryIOException.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/SortedList.java
 Move to hbase-common. Its a generic Interface. Need by

M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/CoprocessorEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContext.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContextImpl.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ReadOnlyConfiguration.java
 Move to hbase-coprocessor.

M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/BigDecimalColumnInterpreter.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/DoubleColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/LongColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/ColumnInterpreter.java
 Moved to hbase-endpoint where they are used.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
 Include region name when toString'd.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCoprocessorHost.java
 Include WAL name when toString'd.
M hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 Add utility used in testing here from CoprocessorHost.
2021-03-31 16:14:45 -07:00
Toshihiro Suzuki 1d3ea38f1e
HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3107)
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 23:04:29 +09:00
Josh Elser fa8b9659b1 HBASE-25692 Always try to close the WAL reader when we catch any exception (#3090)
There are code paths in which we throw non-IOExceptions when
initializing a WAL reader. However, we only close the InputStream to the
WAL filesystem when the exception is an IOException. Close it if it is
open in all cases.

Co-authored-by: Josh Elser <jelser@cloudera.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-03-29 12:17:16 -07:00
Michael Stack 84eb0220a4
HBASE-25695 Link to the filter on hbase:meta from user tables panel on master page (#3092)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-03-27 20:35:20 -07:00
caroliney14 3bb978894d
HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2771)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-25 17:10:43 -07:00
Andrew Purtell 124ea5eee3 HBASE-25693 NPE getting metrics from standby masters (MetricsMasterWrapperImpl.getMergePlanCount) (#3091)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-03-24 19:10:58 -07:00
huaxiangsun 80529af963
HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3087)
The issue is that FileInputStream is created with try-with-resources, so its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected, its close() is called again.
To avoid this double-free resources, add guard against it.

Signed-off-by: stack <stack@apache.org>
2021-03-24 14:38:44 -07:00
Toshihiro Suzuki 2b5c2c36d9
HBASE-25678 Support nonce operations for Increment/Append in RowMutations and CheckAndMutate (#3073)
Signed-off-by: stack <stack@apache.org>
2021-03-21 22:47:53 +09:00
shahrs87 ed8b72627d
HBASE-25679 Size of log queue metric is incorrect (#3071)
Co-authored-by: Rushabh <rushabh.shah@salesforce.com>
Signed-off-by: stack <stack@apache.org>
2021-03-19 16:14:45 -07:00
haxiaolin 566b0512a7 HBASE-25518 Support separate child regions to different region servers (#3001)
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:39:00 -07:00
bsglz 19d094b96f HBASE-25643 The delayed FlushRegionEntry should be removed when we ne… (#3049)
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:13:34 -07:00
Michael Stack 75e2e7c83d HBASE-25677 Server+table counters on each scan #nextRaw invocation becomes a bottleneck when heavy load (#3061)
Don't have every handler update regionserver metrics on each
scan#nextRaw; instead, do a batch update just before Scan
returns. Otherwise, all running handlers end up contending
on metrics update.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Update of regionserver metrics counters moved out to caller where
 can be done as a batch update instead of per-next.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
 Class doc to encourage batch updating metrics.
 Remove the single update as unused anymore.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 Count calls to nextRaw. Update regionserver count in finally block when
 scan is done rather than per nextRaw call. Move all metrics updates to
 finally.

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Baiqiang Zhao <ZhaoBQ>
2021-03-18 11:40:42 -07:00
Sandeep Pal 72496272aa
HBASE-25627: HBase replication should have a metric to represent if the source is stuck getting initialized (#3018)
Introduces a new metric that tracks number of replication sources that are stuck in initialization.

Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit ff3821814a)
2021-03-17 10:30:26 -07:00
bitterfox a61e9b963d HBASE-25665 Option to use hostname instead of canonical hostname for secure HBase cluster connection (#3051) 2021-03-16 21:05:23 -07:00
Baiqiang Zhao a93b0cd3ca HBASE-25597 Add row info in Exception when cell size exceeds maxCellSize (#2976)
Signed-off-by: stack <stack@apache.org>
2021-03-15 15:50:22 -07:00
haxiaolin 6c224b8640 HBASE-25621 Balancer should check region plan source to avoid misplace region groups (#3002)
Signed-off-by: stack <stack@duboce.net>
2021-03-15 14:48:35 -07:00
haxiaolin 13574de156 HBASE-25595 TestLruBlockCache.testBackgroundEvictionThread is flaky (#2974)
Signed-off-by: stack <stack@apache.org>
2021-03-15 14:26:06 -07:00
Michael Stack 3bf2ef4259
HBASE-25660 Print split policy in use on Region open (as well as split policy vitals) (#3044)
Add a toString to all split policy implementations listing name and
 vitals. Use this toString in the Region open message. Ditto for flush
 policy for the Region.

 Signed-off-by: Huaxiang Sun<huaxiangsun@apache.org>
2021-03-15 14:10:37 -07:00
haxiaolin 95e8ee5ca5 HBASE-25635 CandidateGenerator may miss some region balance actions (#3024)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-03-15 21:33:46 +08:00
Duo Zhang aafbbc2456 HBASE-25657 Fix spotbugs warnings after upgrading spotbugs to 4.x (#3041)
Signed-off-by: meiyi <myimeiyi@gmail.com>
Signed-off-by: stack <stack@apache.org>
2021-03-12 14:37:17 +08:00
shahrs87 bb4d8b9cf7 HBASE-25622 Result#compareResults should compare tags. (#3026)
Signed-off-by: stack <stack@apache.org>
2021-03-11 21:52:11 -08:00
Michael Stack 45b3280497 HBASE-25570 On largish cluster, "CleanerChore: Could not delete dir..." makes master log unreadable (#2949)
Turn down the amount we log. If you want to see the full exception
enable TRACE-level logging.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: shahrs87
2021-03-11 21:36:31 -08:00
Peter Somogyi af8dc2e4df HBASE-25587 [hbck2] Schedule SCP for all unknown servers (#2978)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-11 17:22:08 +01:00
meiyi b5fc5e17e2 HBASE-25636 Expose HBCK report as metrics (#3031)
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-03-11 15:15:23 +08:00
Duo Zhang d546c8e77c HBASE-25629 Reimplement TestCurrentHourProvider to not depend on unstable TZs (#3013)
Signed-off-by: XinSun <ddupgs@gmail.com>
2021-03-10 23:53:59 +08:00