Commit Graph

8386 Commits

Author SHA1 Message Date
Andrew Purtell 4eaba9e9b1 Revert "Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md"
This reverts commit c2707bc6a8.
2021-05-19 12:28:57 -07:00
Andrew Purtell 49ed8cc306 Revert "Preparing development version 2.4.4-SNAPSHOT"
This reverts commit cf287aaf95.
2021-05-19 12:28:48 -07:00
Andrew Purtell cf287aaf95 Preparing development version 2.4.4-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-05-19 11:13:54 -07:00
Andrew Purtell c2707bc6a8 Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-05-19 11:13:48 -07:00
Andrew Purtell 97c8201f9a Revert "Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md"
This reverts commit f9ed394627.
2021-05-19 10:52:24 -07:00
Andrew Purtell ed71600540 Revert "Preparing development version 2.4.4-SNAPSHOT"
This reverts commit 231c37b4f6.
2021-05-19 10:52:15 -07:00
Andrew Purtell 231c37b4f6 Preparing development version 2.4.4-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-05-19 10:18:46 -07:00
Andrew Purtell f9ed394627 Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-05-19 10:18:40 -07:00
Andrew Purtell 8ce0aba77e Revert "Preparing hbase release 2.4.3RC0; tagging and updates to CHANGES.md and RELEASENOTES.md"
This reverts commit af14fd998c.
2021-05-19 10:08:39 -07:00
Andrew Purtell d82426c41c Revert "Preparing development version 2.4.4-SNAPSHOT"
This reverts commit 0c5c0e5da8.
2021-05-19 10:07:40 -07:00
Duo Zhang 2bf86468d8 HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-18 10:43:13 +08:00
Pankaj 21c1dc7e6c HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager (#3250)
* HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager's start & retrievePassword method

Signed-off-by: stack <stack@apache.com>
(cherry picked from commit 2126ec94f0)
2021-05-17 22:51:11 +05:30
Andrew Purtell 0c5c0e5da8 Preparing development version 2.4.4-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-05-14 13:12:49 -07:00
Andrew Purtell 13043ecf96 Revert "Preparing development version 2.4.4-SNAPSHOT"
This reverts commit 8fd02a75f8.
2021-05-14 13:12:49 -07:00
Viraj Jasani 112b58ad8c
HBASE-25884 Return empty records for disabled balancer in-memory queue (#3263)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
2021-05-14 12:57:59 +05:30
Michael Stack 1c829af56a HBASE-25876 Add retry if we fail to read all bytes of the protobuf magic marker (#3251)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2021-05-12 19:23:14 +01:00
Michael Stack 924f256006
HBASE-25867 Extra doc around ITBLL (#3242)
* HBASE-25867 Extra doc around ITBLL

Minor edits to a few log messages.
Explain how the '-c' option works when passed to ChaosMonkeyRunner.
Some added notes on ITBLL.
Fix whacky 'R' and 'Not r' thing in Master (shows when you run ITBLL).
In HRS, report hostname and port when it checks in (was debugging issue
where Master and HRS had different notions of its hostname).
Spare a dirty FNFException on startup if base dir not yet in place.

* Address Review by Sean

Signed-off-by: Sean Busbey <busbey@apache.org>
2021-05-11 11:23:14 -07:00
Catalin Luca 992d80ad39 HBASE-25859 Reference class incorrectly parses the protobuf magic marker (#3236)
Co-authored-by: Catalin Luca <luca@adobe.com>
Signed-off-by: stack <stack@apache.org>
2021-05-10 14:50:50 -05:00
Viraj Jasani e4dc6eaa76
HBASE-25790 Test fix Addendum 2021-05-09 00:39:31 +05:30
GeorryHuang 26e9c5b3ce
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) (#3245)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-09 00:26:58 +05:30
Rushabh Shah c33767029e
HBASE-25860 Add metric for successful wal roll requests. (#3238)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-08 13:08:34 +05:30
Andrew Purtell 6978b8a0ef HBASE-25774 ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
Revert "HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2771)"

This reverts commit 5d7e2f71f7.
2021-05-07 17:56:57 -07:00
Andrew Purtell 4de3841074 HBASE-25854 Remove redundant AM in-memory state changes in CatalogJanitor (#3234)
In CatalogJanitor we schedule GCRegionProcedure to clean up both
filesystem and in-memory state after a split, and
GCMultipleMergedRegionsProcedure to do the same for merges. Both of these
procedures clean up in-memory state, but CatalogJanitor also does this
redundantly just after scheduling the procedures. The cleanup should be
done in only one place. Presumably we are using the procedures to do it in
a principled way. Remove the redundancy in CatalogJanitor and fix any
follow on issues, like test failures.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-06 09:17:35 -07:00
Andrew Purtell c32b6c8eff HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore (#3230)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-05 17:02:18 -07:00
Andrew Purtell 0fc6852f6a HBASE-25835 Ignore duplicate split requests from regionserver reports (#3218)
Processing of the RS report happens asynchronously from other activities
which can mutate region state. For example, a split procedure may already
be running. A split procedure cannot succeed if the parent region is no
longer open, so we can ignore it in that case.

Note that submitting more than one split procedure for a given region is
harmless -- the split is fenced in the procedure handling -- but it would
be noisy in the logs. Only one procedure can succeed. The other
procedure(s) would abort during initialization and report failure with
WARN level logging.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2021-05-04 10:23:04 -07:00
Andrew Purtell fd3e28364c HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions (#3219)
RegionStates#getAssignmentsForBalancer is used by the HMaster to
collect all regions of interest to the balancer for the next chore
iteration. We check if a table is in disabled state to exclude
regions that will not be of interest (because disabled regions are
or will be offline) or are in a state where they shouldn't be
mutated (like SPLITTING). The current checks are not actually
comprehensive.

Filter out regions not in OPEN or OPENING state when building the
set of interesting regions for the balancer to consider. Only
regions open (or opening) on the cluster are of interest to
balancing calculations for the current iteration. Regions in all
other states can be expected to not be of interest – either offline
(OFFLINE, or FAILED_*), not subject to balancer decisions now
(SPLITTING, SPLITTING_NEW, MERGING, MERGING_NEW), or will be
offline shortly (CLOSING) – until at least the next chore
iteration.

Add TRACE level logging.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-03 18:24:17 -07:00
Andrew Purtell 368d52fe99 HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip (#3223)
We claim in a WARN level log line to be "Playing-it-safe skipping merge/
split gc'ing of regions from hbase:meta while regions-in-transition (RIT)"
but do not actually skip because of a missing return. Remove the warning.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-03 18:17:56 -07:00
Duo Zhang 08507b86a8 HBASE-25838 Use double instead of Double in StochasticLoadBalancer (#3221)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-04 09:10:35 +08:00
Andrew Purtell 8fd02a75f8 Preparing development version 2.4.4-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-04-27 14:58:08 -07:00
Andrew Purtell af14fd998c Preparing hbase release 2.4.3RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-04-27 14:58:03 -07:00
stack 12200e9d89 HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars (#3184)
Need to add to allowed-licenses list too....

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-04-27 09:07:21 -07:00
Duo Zhang e765e19279 HBASE-25806 Backport the region location finder initialization fix in HBASE-25802 (#3196)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-26 13:07:41 +08:00
haxiaolin 1dc881ab85 HBASE-25754 StripeCompactionPolicy should support compacting cold regions (#3152)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-23 15:32:27 +08:00
Toshihiro Suzuki cc7c4352ee HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-04-22 14:14:36 +09:00
Duo Zhang a15fd5cff5 HBASE-25775 Use a special balancer to deal with maintenance mode (#3161)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-04-16 13:40:29 +08:00
Nick Dimiduk 7179c013bb HBASE-25770 Http InfoServers should honor gzip encoding when requested (#3159)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-04-15 13:53:01 -07:00
xiaoyu 22fd7d7f10
HBASE-25776 Use Class.asSubclass to fix the warning in StochasticLoadBalancer.loadCustomCostFunctions (#3163)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-15 23:37:05 +05:30
Duo Zhang 69e93c8137 HBASE-25767 CandidateGenerator.getRandomIterationOrder is too slow on large cluster (#3149)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-14 08:29:01 +08:00
DivyeshChandra 082ad7cf21 HBASE-25653 Add units and round off region size to 2 digits after decimal (#3046)
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-04-13 09:07:19 -07:00
Duo Zhang d2cba187e5 HBASE-25759 The master services field in LocalityBasedCostFunction is never used (#3144)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-12 22:43:48 +08:00
Geoffrey Jacoby 843e1369f7 HBASE-25751 - Add writable TimeToPurgeDeletes to ScanOptions (#3137)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-09 13:09:28 -07:00
meiyi da6b1df643 HBASE-25747 Remove unused getWriteAvailable method in OperationQuota (#3133)
Signed-off-by: stack <stack@apache.org>
2021-04-09 10:26:26 +08:00
Pankaj e2dedf9ef1
HBASE-25717 RegionServer aborted with due to ClassCastException (#3108)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-08 16:30:12 +05:30
Jan Hentschel 41b4e6e5c4
HBASE-24305 Prepare deprecations in ServerName (#1666) (#3125)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-04-06 19:10:52 +02:00
Jan Hentschel 7b0af17551 HBASE-25199 Remove deprecated HStore#getStoreHomedir methods (#2562)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-03 23:44:37 +08:00
d-c-manning 8e49848e75 HBASE-25726 MoveCostFunction is not included in the list of cost functions for StochasticLoadBalancer (#3116)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-04-02 10:45:49 -07:00
lujiefsi 3234154689 HBASE-25558:Adding audit log for execMasterService (#3101)
Signed-off-by: stack <stack@apache.org>
2021-03-31 16:16:59 -07:00
Toshihiro Suzuki 6f4ab4a134 HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3107)
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 23:06:04 +09:00
Josh Elser 92f7be5da7 HBASE-25692 Always try to close the WAL reader when we catch any exception (#3090)
There are code paths in which we throw non-IOExceptions when
initializing a WAL reader. However, we only close the InputStream to the
WAL filesystem when the exception is an IOException. Close it if it is
open in all cases.

Co-authored-by: Josh Elser <jelser@cloudera.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-03-29 12:20:06 -07:00
Michael Stack c1bda19694 HBASE-25695 Link to the filter on hbase:meta from user tables panel on master page (#3092)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-03-27 20:36:08 -07:00