5323 Commits

Author SHA1 Message Date
Duo Zhang
72d7641fb5
HBASE-25852 Move all the intialization work of LoadBalancer implementation to initialize method (#3248) (#3258)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-13 14:19:03 +08:00
GeorryHuang
63d49cb7ed
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) (#3245)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-09 00:09:23 +05:30
Rushabh Shah
90dc150b1b
HBASE-25860 Add metric for successful wal roll requests. (#3238)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-08 13:02:52 +05:30
Andrew Purtell
d60aa0a865 HBASE-25774 ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
Revert "HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2771)"

This reverts commit 3bb978894db2c7ce41010e403a83a8a9f5d1110e.
2021-05-07 17:55:20 -07:00
niuyulin
5e65da64e2 HBASE-25837 TestRollingRestart is flaky (#3220)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-07 20:28:37 +08:00
Andrew Purtell
2938ba421b HBASE-25854 Remove redundant AM in-memory state changes in CatalogJanitor (#3234)
In CatalogJanitor we schedule GCRegionProcedure to clean up both
filesystem and in-memory state after a split, and
GCMultipleMergedRegionsProcedure to do the same for merges. Both of these
procedures clean up in-memory state, but CatalogJanitor also does this
redundantly just after scheduling the procedures. The cleanup should be
done in only one place. Presumably we are using the procedures to do it in
a principled way. Remove the redundancy in CatalogJanitor and fix any
follow on issues, like test failures.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-06 09:17:29 -07:00
Duo Zhang
2d4904ceea
HBASE-25851 Make LoadBalancer not extend Configurable interface (#3233) (#3235)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-06 23:39:43 +08:00
Nick Dimiduk
6f2842d1af HBASE-25843 move master http-related code into o.a.h.h.master.http
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-05 08:36:40 -07:00
Duo Zhang
afa9836b87 HBASE-25834 Remove balanceTable method from LoadBalancer interface (#3217)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-05 16:03:51 +08:00
Nick Dimiduk
3f8d3ba6d9 HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-04 15:46:53 -07:00
Duo Zhang
c52c091609 HBASE-25819 Fix style issues for StochasticLoadBalancer (#3207)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-29 11:26:58 +08:00
Nick Dimiduk
3a01941963 HBASE-25779 HRegionServer#compactSplitThread should be private
Minor refactor. Make the `compactSplitThread` member field of `HRegionServer` private, and gate
all access through the getter method.

Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-04-28 16:54:44 -07:00
Duo Zhang
f374357cc3 HBASE-25802 Miscellaneous style improvements for load balancer related classes (#3192)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-23 16:32:10 +08:00
haxiaolin
fd7b2c6eec HBASE-25754 StripeCompactionPolicy should support compacting cold regions (#3152)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-23 15:31:55 +08:00
Duo Zhang
a02ce95ff1
HBASE-25793 Move BaseLoadBalancer.Cluster to a separated file (#3185) (#3190)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-23 00:08:16 +08:00
Toshihiro Suzuki
f4f84302fa HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-04-22 14:06:28 +09:00
Duo Zhang
a177fb4c62 HBASE-25775 Use a special balancer to deal with maintenance mode (#3161)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-04-16 13:39:48 +08:00
Duo Zhang
702220a135 HBASE-25759 The master services field in LocalityBasedCostFunction is never used (#3144)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-12 22:31:05 +08:00
Geoffrey Jacoby
89c9f28d3f HBASE-25751 - Add writable TimeToPurgeDeletes to ScanOptions (#3137)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-09 13:09:24 -07:00
Baiqiang Zhao
8ff17c68e2
HBASE-25687 Backport "HBASE-25681 Add a switch for server/table query… (#3074)
Signed-off-by: stack <stack@apache.org>
2021-04-07 11:11:46 -07:00
Jan Hentschel
18882d6f9a
HBASE-24305 Prepare deprecations in ServerName (#1666) (#3121)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-04-05 22:10:04 +02:00
Jan Hentschel
1fa07cea80 HBASE-25199 Remove deprecated HStore#getStoreHomedir methods (#2562)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-03 23:44:23 +08:00
d-c-manning
554e039817 HBASE-25726 MoveCostFunction is not included in the list of cost functions for StochasticLoadBalancer (#3116)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-04-02 10:43:51 -07:00
stack
b051ad5594 Revert "HBASE-25713 Make an hbase-wal module"
This reverts commit e58f1b24a6c1f3aaa511a71b2ac77280921fa210.
2021-03-31 16:15:50 -07:00
stack
e58f1b24a6 HBASE-25713 Make an hbase-wal module
First extract an hbase-coprocessor module used by hbase-client, hbase-server.
This is prerequisite to extracting an hbase-wal module.

M hbase-common/src/main/java/org/apache/hadoop/hbase/Abortable.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/DoNotRetryIOException.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/SortedList.java
 Move to hbase-common. Its a generic Interface. Need by

M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/CoprocessorEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContext.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContextImpl.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ReadOnlyConfiguration.java
 Move to hbase-coprocessor.

M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/BigDecimalColumnInterpreter.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/DoubleColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/LongColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/ColumnInterpreter.java
 Moved to hbase-endpoint where they are used.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
 Include region name when toString'd.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCoprocessorHost.java
 Include WAL name when toString'd.
M hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 Add utility used in testing here from CoprocessorHost.
2021-03-31 16:14:45 -07:00
Toshihiro Suzuki
1d3ea38f1e
HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3107)
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 23:04:29 +09:00
Josh Elser
fa8b9659b1 HBASE-25692 Always try to close the WAL reader when we catch any exception (#3090)
There are code paths in which we throw non-IOExceptions when
initializing a WAL reader. However, we only close the InputStream to the
WAL filesystem when the exception is an IOException. Close it if it is
open in all cases.

Co-authored-by: Josh Elser <jelser@cloudera.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-03-29 12:17:16 -07:00
caroliney14
3bb978894d
HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2771)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-25 17:10:43 -07:00
Toshihiro Suzuki
2b5c2c36d9
HBASE-25678 Support nonce operations for Increment/Append in RowMutations and CheckAndMutate (#3073)
Signed-off-by: stack <stack@apache.org>
2021-03-21 22:47:53 +09:00
shahrs87
ed8b72627d
HBASE-25679 Size of log queue metric is incorrect (#3071)
Co-authored-by: Rushabh <rushabh.shah@salesforce.com>
Signed-off-by: stack <stack@apache.org>
2021-03-19 16:14:45 -07:00
haxiaolin
566b0512a7 HBASE-25518 Support separate child regions to different region servers (#3001)
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:39:00 -07:00
bsglz
19d094b96f HBASE-25643 The delayed FlushRegionEntry should be removed when we ne… (#3049)
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:13:34 -07:00
Sandeep Pal
72496272aa
HBASE-25627: HBase replication should have a metric to represent if the source is stuck getting initialized (#3018)
Introduces a new metric that tracks number of replication sources that are stuck in initialization.

Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit ff3821814aa79248ec82f1c07f9d624e7dfb2334)
2021-03-17 10:30:26 -07:00
bitterfox
a61e9b963d HBASE-25665 Option to use hostname instead of canonical hostname for secure HBase cluster connection (#3051) 2021-03-16 21:05:23 -07:00
haxiaolin
95e8ee5ca5 HBASE-25635 CandidateGenerator may miss some region balance actions (#3024)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-03-15 21:33:46 +08:00
shahrs87
bb4d8b9cf7 HBASE-25622 Result#compareResults should compare tags. (#3026)
Signed-off-by: stack <stack@apache.org>
2021-03-11 21:52:11 -08:00
Peter Somogyi
af8dc2e4df HBASE-25587 [hbck2] Schedule SCP for all unknown servers (#2978)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-11 17:22:08 +01:00
Duo Zhang
d546c8e77c HBASE-25629 Reimplement TestCurrentHourProvider to not depend on unstable TZs (#3013)
Signed-off-by: XinSun <ddupgs@gmail.com>
2021-03-10 23:53:59 +08:00
Wellington Ramos Chevreuil
7adc00044c HBASE-25548 Optionally allow snapshots to preserve cluster's max file… (#2923)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 373dc7788d66ba7f7c49d727373c521c639d613c)
2021-03-10 10:35:58 +00:00
Anoop Sam John
29d771fab9
HBASE-25582 Support setting scan ReadType to be STREAM at cluster level (#3040)
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-03-10 13:23:15 +05:30
Bharath Vissapragada
321b5c7979
HBASE-25547: Thread pools should release unused resources (#3037)
* HBASE-25547: Thread pools should release unused resources (#2922)

Plumbs the configuration needed to enable core thread timeout on non-critical thread pools.
Currently only enabled for thread pools with op-codes RS_LOG_REPLAY_OPS, RS_PARALLEL_SEEK, MASTER_SNAPSHOT_OPERATIONS, MASTER_MERGE_OPERATIONS. Others can be added later as
needed.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 618236dd9040625cf38a4d98338ccff8ab35bfec)

* HBASE-25547 (addendum): Roll ExecutorType into ExecutorConfig

Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 4c822d7463589796472725524be4fa2509c7bcef)
2021-03-09 09:03:16 -08:00
niuyulin
88b9e98a7d HBASE-25637 Rename method completeCompaction to refreshStoreSizeAndTotalBytes (#3023)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-03-06 18:43:47 +08:00
Rahul Kumar
e57c73a137
HBASE-25460 : Expose drainingServers as cluster metric (#2994) (#2995)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-03-04 12:48:57 +05:30
Andrew Purtell
dce639ae81 HBASE-25385 TestCurrentHourProvider fails if the latest timezone changes are not present (#3012)
Disable this problematic test for now. May be removed after additional
discussion.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-03-03 18:16:20 -08:00
niuyulin
621edfe1da HBASE-25603 Add switch for compaction after bulkload (#2982)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-04 10:11:19 +08:00
Sandeep Pal
80b835927c
HBASE-25596: Fix NPE and avoid permanent unreplicated data due to EOF (#3008)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-03-03 11:38:46 -08:00
Andrew Purtell
0849fae730 Revert "HBASE-25596: Fix NPE and avoid permanent unreplicated data due to EOF (#2990)"
This reverts commit d724d0576fcafb507b32d3f6636228d559a924fa.
2021-03-02 12:26:23 -08:00
XinSun
328ff8c05a HBASE-25590 Bulkload replication HFileRefs cannot be cleared in some cases where set exclude-namespace/exclude-table-cfs (#2969)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-02-26 09:58:05 +08:00
Sandeep Pal
d724d0576f
HBASE-25596: Fix NPE and avoid permanent unreplicated data due to EOF (#2990)
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-02-25 13:36:31 -08:00
XinSun
e863df383b HBASE-25598 TestFromClientSide5.testScanMetrics is flaky (#2977)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-02-24 14:19:36 +08:00