Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,989 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

8594 Commits

Author SHA1 Message Date
Andrew Purtell 4149459dde HBASE-26383 HBCK incorrectly reports inconsistencies for recently split regions following a master failover (#3775) 
A regression was introduced by HBASE-25847 which changed regionInfo#isParentSplit
to regionState#isSplit. The region state after restart is CLOSED instead of SPLIT.
We need to check both regionState and regionInfo for split status.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-10-20 15:55:10 -07:00
Viraj Jasani 09ffa49df8
HBASE-26371 Prioritize meta region move over other region moves in region_mover (#3769) (#3767) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-10-20 23:07:28 +05:30
Josh Elser a372d437b3 HBASE-26350 Add a DEBUG when we fail the SASL handshake 
Closes #3743

Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-10-14 15:42:53 -04:00
Peter Somogyi c42d6ccc1a
HBASE-26329 Upgrade commons-io to 2.11.0 (#3735) 
Includes HBASE-25451 changes required for commons-io 2.8.0+ upgrade.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-10-10 09:47:57 +02:00
Andrew Purtell 2ca84d531a Preparing development version 2.4.8-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-10-08 17:10:33 -07:00
Andrew Purtell dad725bbe2 Preparing hbase release 2.4.7RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-10-08 17:10:27 -07:00
chenglei 3cc539a561 HBASE-26295 BucketCache could not free BucketEntry which restored fro… (#3699) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-29 21:47:11 +08:00
richardantal 43455061d3 HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results (#3688) 
Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-29 10:23:43 +02:00
bitterfox a6329385a0 HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME (#3647) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-28 23:16:33 +08:00
clarax dc2bb6a908 HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison (#3698) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-26 23:45:37 +08:00
Andrew Purtell 8bd4b2c9f3 Amend HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job 
Compliation fixes for CombinedBlockCache and TestClientSideRegionScanner.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-09-22 12:20:00 -07:00
Tak Lon (Stephen) Wu 6ca495b273 HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job (#3684) 
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching
INDEX/LEAF_INDEX block when a client, e.g.
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.

Signed-off-by: Josh Elser <elserj@apache.org>
 2021-09-22 09:39:38 -07:00
meiyi 6f68d2765c HBASE-26261 Store configuration loss when use update_config (#3664) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-22 19:08:07 +08:00
meiyi d75d73bcc7 HBASE-26270 Provide getConfiguration method for Region and Store interface (#3671) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-09-22 11:24:19 +08:00
chenglei 9414e3b471 HBASE-26281 DBB got from BucketCache would be freed unexpectedly before RPC completed (#3680) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-17 22:29:18 +08:00
Xiaolin Ha 466d2d5563 HBASE-26229 Limit count and size of L0 files compaction in StripeCompactionPolicy (#3646) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-09 18:30:24 +08:00
hapihu 9e04033ad2 [HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java" (#3650) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-09 12:22:19 +08:00
Andrew Purtell 24971e8073 Preparing development version 2.4.7-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-09-03 09:41:58 -07:00
Andrew Purtell 7374d396c2 Preparing hbase release 2.4.6RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-09-03 09:41:52 -07:00
Rushabh Shah f9a9b6eaee HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
 2021-09-02 12:08:49 -07:00
XinSun 067f609ec6 HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit 345d7256c8)
 2021-09-02 15:33:24 +08:00
chenglei cf4c764d16 HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 18:17:26 +08:00
Andrew Purtell 122efbcd82 HBASE-25642 Fix or stop warning about already cached block (#3638) 
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)

Because it is harmless, log at DEBUG level, not WARN.

Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
 2021-08-31 09:26:14 -07:00
keyserbrian1 133237582f HBASE-26232 SEEK_NEXT_USING_HINT is ignored on reversed Scans (#3639) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-29 21:43:02 +08:00
Andrew Purtell 21e2c1b7c6 HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG (#3579) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-08-27 10:15:19 -07:00
Pankaj 1a5b9b7ff6 HBASE-26227 Forward port HBASE-26223 test code to branch-2.4+ (#3632) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-27 16:47:06 +08:00
lpalasthy da74b8c8fb HBASE-26219 Negative time is logged while waiting on regionservers (#3622) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-25 16:40:16 +02:00
Xiaolin Ha d6b2257655 HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler (#3489) 
Signed-off-by: stack <stack@apache.org>
 2021-08-18 14:41:56 +08:00
eomiks 96015e9669 HBASE-24652 master-status UI make date type fields sortable 2021-08-16 16:02:52 -07:00
stack 2ad7905ddc HBASE-26200 Revert "HBASE-25165 Change 'State time' in UI so sorts (#2508)" 
This reverts commit d07d181ea4.
 2021-08-16 13:46:05 -07:00
lzx404243 627e0d61ba HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner (#2984) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-14 11:11:46 +08:00
chenglei 8fced2d7f8 HBASE-26026 HBase Write may be stuck forever when using CompactingMemStore (#3421) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 23:22:09 +08:00
Xiaolin Ha 2d89b08eb0 HBASE-26155 JVM crash when scan (#3553) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 17:20:18 +08:00
Rushabh Shah 9d34fa7d58
HBASE-26185 Return mutable list in AssignmentManager#getExcludedServersForSystemTable (#3577) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-08-12 13:48:17 +05:30
Viraj Jasani cd007eb6b2
HBASE-26185 Fix TestMaster#testMoveRegionWhenNotInitialized in the presence of hbase.min.version.move.system.tables (#3573) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-08-10 10:59:14 +05:30
Richard Marscher 66f931c433 HBASE-6908: Add pluggable rpc queue implementation (#3522) 
Can pass in a FQCN to load as the call queue implementation.

Standardized arguments to the constructor are the max queue length, the
PriorityFunction, and the Configuration.

`PluggableBlockingQueue` abstract class provided to help guide the
correct constructor signature

Hard fails if the class fails to load as a `BlockingQueue<CallRunner>`

Signed-off-by: stack <stack@apache.org>
 2021-08-09 12:56:25 -07:00
Duo Zhang 3182507652 HBASE-26179 TestRequestTooBigException spends too much time to finish (#3571) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-08-07 23:08:10 +08:00
Xiaolin Ha 2ea2f800b0 HBASE-26170 handleTooBigRequest in NettyRpcServer didn't skip enough bytes (#3564) 
Signed-off-by: stack <stack@apache.com>
 2021-08-05 08:18:09 -07:00
chenglei 9a469aeaae HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero (#3531) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-05 21:04:57 +08:00
caroliney14 428cff499d HBASE-25469 Add detailed RIT info in JSON format for consumption as metrics (#3555) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 18:22:49 -07:00
Andrew Purtell 91130af844 Revert "Add detailed RIT info in JSON format for consumption as metrics (#3555)" 
Commit is missing JIRA identifier.

This reverts commit 9398745f44.
 2021-08-04 18:22:30 -07:00
caroliney14 9398745f44 Add detailed RIT info in JSON format for consumption as metrics (#3555) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 18:07:25 -07:00
Ruanhui 348c39236c HBASE-26166 Fix a minor bug of table list on master ui (#3554) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-08-04 20:41:11 +08:00
Bryan Beaudreault 63b17d1674
HBASE-26154: Adds exception metrics for QuotaExceededException and RpcThrottlingException (#3546) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-08-02 15:19:59 +05:30
YutSean cb1174e9e1 HBASE-26105 Rectify the expired TODO comment in CombinedBC. (#3509) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-30 21:43:38 +08:00
Andrew Purtell 4ec1333df8 Preparing development version 2.4.6-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-27 09:34:19 -07:00
Andrew Purtell 03b8c0cf42 Preparing hbase release 2.4.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-27 09:34:14 -07:00
Duo Zhang eb4728116d HBASE-26120 New replication gets stuck or data loss when multiwal groups more than 10 (#3528) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-27 08:15:48 -07:00
Andrew Purtell 845e6f0216 Preparing development version 2.4.6-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-26 17:24:44 -07:00
Andrew Purtell 4810fd1096 Preparing hbase release 2.4.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-26 17:24:37 -07:00
Andrew Purtell 67163bb1da Preparing development version 2.4.6-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-23 09:55:58 -07:00
Andrew Purtell 4a2828386f Preparing hbase release 2.4.5RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-23 09:55:52 -07:00
Rushabh Shah 9a8fff78e7 HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory (#3504) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-23 09:33:50 -07:00
meiyi a3940afd25 HBASE-24734 RegionInfo#containsRange should support check meta table (#3496) (#3517) 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2021-07-22 18:05:14 +08:00
gkanade a72c62b022 HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation (#3512) 
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-07-22 08:35:46 +05:30
Aman Poonia 02c796ca0e
HBASE-25986 set default value of normalization enabled from hbase site (#3476) (#3372) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-07-19 15:29:02 +05:30
clarax 6ab6d6f231
HBASE-25973 Balancer should explain progress in a better way in log - backport branch-2 (#3485) 
Signed-off-by: stack <stack@apache.org>
 2021-07-16 15:20:14 -07:00
clarax dd2ae3605d
HBASE-25739 TableSkewCostFunction need to use aggregated deviation - … (#3481) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
    Signed-off-by: stack <stack@duboce.net>
    Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-07-16 10:53:41 -07:00
YutSean 5ee1447b55 HBASE-26083 In branch-2 & branch-1 L1 miss metric is always 0 when using CombinedBlockCache (#3473) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Reid Chan <reidchan@apache.org>
 2021-07-16 10:38:05 +08:00
binlijin 1d0cfa9ed4 HBASE-26086 TestHRegionReplayEvents do not pass in branch-2 and throws NullPointerException (#3480) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-07-15 09:53:34 +08:00
Xiaolin Ha b83fec48dc
HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations (#3436) (#3486) (#3487) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-14 16:40:25 +08:00
Duo Zhang 273c5c567d HBASE-26068 The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong (#3461) 
Signed-off-by: Yulin Niu <niuyulin@apache.org
 2021-07-07 14:39:14 +08:00
Viraj Jasani 4bd7aed064
HBASE-22923 Consider minVersionToMoveSysTables while moving region and creating regionPlan (ADDENDUM) (#3455) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-03 22:45:17 +05:30
Viraj Jasani c50867348b
HBASE-22923 min version of RegionServer to move system table regions (#3439) (#3438) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-01 16:24:47 +05:30
Viraj Jasani 5b5c92f427
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade (#3417) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-01 15:14:23 +05:30
bsglz 399b58e7c6 HBASE-26028 The view as json page shows exception when using TinyLfuBlockCache (#3420) 
(cherry picked from commit 147b030c1f)
 2021-07-01 10:36:20 +08:00
Duo Zhang c5a6e19be0 HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 (#3440) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-30 08:32:36 +08:00
GeorryHuang 270b3facce HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… (#3373) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:33:48 +08:00
GeorryHuang 414deeea83 HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… (#3406) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:13:36 +08:00
YutSean 3a6441c946 HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 (#3411) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-24 15:37:46 +08:00
Duo Zhang b453cbd819 HBASE-26020 Split TestWALEntryStream.testDifferentCounts out (#3409) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-06-23 23:19:28 +08:00
litao 8d1473d321
HBASE-25934 Add username for RegionScannerHolder (#3325) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-23 12:55:21 +05:30
lujiefsi f09924b650 HBASE-25877 Add access check for compactionSwitch (#3253) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 23:47:04 +08:00
belugabehr ea439c652a HBASE-25937: Clarify UnknownRegionException (#3330) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 22:44:21 +08:00
Duo Zhang ad545aa548 HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back 2021-06-21 23:59:11 +08:00
YutSean 02714cbc18 HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid (#3403) 
Signed-off-by: Reid Chan <reidchan@apache.org>
(cherry-picked from 029dc81b39)
 2021-06-21 15:30:20 +08:00
Viraj Jasani 0c1d1d26a8
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache (#3215) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-21 12:57:48 +05:30
Duo Zhang beaeb0dfec HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 (#3376) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-20 16:44:51 +08:00
Bharath Vissapragada 58f067dc4d
HBASE-25998: Redo synchronization in SyncFuture (#3400) 
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch

- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb59642)
 2021-06-18 10:37:01 -07:00
Bharath Vissapragada a4ceabdc76
HBASE-25984: Avoid premature reuse of sync futures in FSHLog (#3371) (#3393) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 5a19bcfa98)
 2021-06-17 12:28:18 -07:00
binlijin 814b196f03 HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… (#3380) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 14:42:26 +08:00
Toshihiro Suzuki b061d477a8 HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (#3386) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 13:57:25 +09:00
Duo Zhang 5159eaec12 HBASE-25995 Change the method name for DoubleArrayCost.setCosts (#3381) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-15 12:06:02 +08:00
Duo Zhang 711b76c24f
HBASE-25947 Backport 'HBASE-25894 Improve the performance for region load and region count related cost functions' to branch-2.4 and branch-2.3 (#3339) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-06-11 09:33:08 +08:00
Zheng Wang 1fd584ed2f HBASE-25967 The readRequestsCount does not calculate when the outResults is empty 2021-06-10 11:08:30 +08:00
Xiaolin Ha 7436a345be HBASE-25981 JVM crash when displaying RegionServer UI (#3364) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-09 18:18:48 +08:00
Andrew Purtell 09eebe5cd0 Preparing development version 2.4.5-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-06-07 15:21:44 -07:00
Andrew Purtell 20e7ba45b0 Preparing hbase release 2.4.4RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-06-07 15:21:39 -07:00
Duo Zhang 6d1d64caf7 HBASE-25963 HBaseCluster should be marked as IA.Public (#3348) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-06 21:27:18 +08:00
meiyi 2dd7ec9d9e HBASE-25929 RegionServer JVM crash when compaction (#3318) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-03 17:34:53 +08:00
Bharath Vissapragada 8a41cb7f1d
HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3332) (#3346) 
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c7786)
 2021-06-01 08:04:02 -07:00
Michael Stack a43afa3576 HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO (#3334) 
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-29 09:10:23 -07:00
Sandeep Pal d37f490d40
HBASE-25927: Fix the log messages by not stringifying the exceptions in log (#3338) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 9a2027bf71)
 2021-05-28 21:35:44 -07:00
Duo Zhang 042aa14fb7 HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one (#3329) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-29 11:11:06 +08:00
Victor 296be5b426
HBASE-25910 - Fix port assignment test (#3308) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-28 20:53:42 +05:30
Wellington Ramos Chevreuil fa577ad92c HBASE-25933 Log trace raw exception, instead of cause message in NettyRpcServerRequestDecoder (#3323) 
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Cherry picked from commit feb89d988b)
 2021-05-27 20:04:42 +01:00
Rushabh Shah 286899450d HBASE-25924 Re-compute size of WAL file while removing from WALEntryStream (#3315) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-05-26 10:43:48 -07:00
Pankaj 22c26f1fc8 HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager (#3250) 
* HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager's start & retrievePassword method

Signed-off-by: stack <stack@apache.com>
(cherry picked from commit 2126ec94f0)
 2021-05-26 00:14:32 +05:30
GeorryHuang 4e079ace58
HBASE-25906 UI of master-status to show recent history of balancer desicion (#3303) (#3296) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-25 12:42:01 +05:30
Anoop Sam John 0df3deab92 HBASE-25898 RS getting aborted due to NPE in Replication WALEntryStream (#3292) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-05-24 23:46:26 +05:30
GeorryHuang a7e6cac7a0 HBASE-25791 UI of master-status to show a recent history of that why balancer was rejected to run (#3278) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-24 21:40:13 +08:00
caoliqing 1e1ac9eda3 HBASE-25892: 'False' should be 'True' in auditlog of listLabels (#3273) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-22 17:25:51 +08:00
Sandeep Pal 646dbfee03
HBASE-25848: Add flexibility to backup replication in case replication filter throws an exception (#3289) 2021-05-21 11:55:47 -07:00
Rushabh Shah 700499be36
HBASE-25827 Per Cell TTL tags get duplicated with increments causing tags length overflow (#3290)(#3210) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-21 23:00:52 +05:30
Andrew Purtell e82340ec64 Preparing development version 2.4.4-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-05-19 12:38:15 -07:00
Andrew Purtell 401b60b217 Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-05-19 12:38:07 -07:00
Andrew Purtell bf518cff56 Revert "HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager (#3250)" 
This reverts commit 21c1dc7e6c.
 2021-05-19 12:29:24 -07:00
Andrew Purtell 2fec951303 Revert "HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268)" 
This reverts commit 2bf86468d8.
 2021-05-19 12:29:12 -07:00
Andrew Purtell 4eaba9e9b1 Revert "Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md" 
This reverts commit c2707bc6a8.
 2021-05-19 12:28:57 -07:00
Andrew Purtell 49ed8cc306 Revert "Preparing development version 2.4.4-SNAPSHOT" 
This reverts commit cf287aaf95.
 2021-05-19 12:28:48 -07:00
Andrew Purtell cf287aaf95 Preparing development version 2.4.4-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-05-19 11:13:54 -07:00
Andrew Purtell c2707bc6a8 Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-05-19 11:13:48 -07:00
Andrew Purtell 97c8201f9a Revert "Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md" 
This reverts commit f9ed394627.
 2021-05-19 10:52:24 -07:00
Andrew Purtell ed71600540 Revert "Preparing development version 2.4.4-SNAPSHOT" 
This reverts commit 231c37b4f6.
 2021-05-19 10:52:15 -07:00
Andrew Purtell 231c37b4f6 Preparing development version 2.4.4-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-05-19 10:18:46 -07:00
Andrew Purtell f9ed394627 Preparing hbase release 2.4.3RC1; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-05-19 10:18:40 -07:00
Andrew Purtell 8ce0aba77e Revert "Preparing hbase release 2.4.3RC0; tagging and updates to CHANGES.md and RELEASENOTES.md" 
This reverts commit af14fd998c.
 2021-05-19 10:08:39 -07:00
Andrew Purtell d82426c41c Revert "Preparing development version 2.4.4-SNAPSHOT" 
This reverts commit 0c5c0e5da8.
 2021-05-19 10:07:40 -07:00
Duo Zhang 2bf86468d8 HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-18 10:43:13 +08:00
Pankaj 21c1dc7e6c HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager (#3250) 
* HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager's start & retrievePassword method

Signed-off-by: stack <stack@apache.com>
(cherry picked from commit 2126ec94f0)
 2021-05-17 22:51:11 +05:30
Andrew Purtell 0c5c0e5da8 Preparing development version 2.4.4-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-05-14 13:12:49 -07:00
Andrew Purtell 13043ecf96 Revert "Preparing development version 2.4.4-SNAPSHOT" 
This reverts commit 8fd02a75f8.
 2021-05-14 13:12:49 -07:00
Viraj Jasani 112b58ad8c
HBASE-25884 Return empty records for disabled balancer in-memory queue (#3263) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
 2021-05-14 12:57:59 +05:30
Michael Stack 1c829af56a HBASE-25876 Add retry if we fail to read all bytes of the protobuf magic marker (#3251) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2021-05-12 19:23:14 +01:00
Michael Stack 924f256006
HBASE-25867 Extra doc around ITBLL (#3242) 
* HBASE-25867 Extra doc around ITBLL

Minor edits to a few log messages.
Explain how the '-c' option works when passed to ChaosMonkeyRunner.
Some added notes on ITBLL.
Fix whacky 'R' and 'Not r' thing in Master (shows when you run ITBLL).
In HRS, report hostname and port when it checks in (was debugging issue
where Master and HRS had different notions of its hostname).
Spare a dirty FNFException on startup if base dir not yet in place.

* Address Review by Sean

Signed-off-by: Sean Busbey <busbey@apache.org>
 2021-05-11 11:23:14 -07:00
Catalin Luca 992d80ad39 HBASE-25859 Reference class incorrectly parses the protobuf magic marker (#3236) 
Co-authored-by: Catalin Luca <luca@adobe.com>
Signed-off-by: stack <stack@apache.org>
 2021-05-10 14:50:50 -05:00
Viraj Jasani e4dc6eaa76
HBASE-25790 Test fix Addendum 2021-05-09 00:39:31 +05:30
GeorryHuang 26e9c5b3ce
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) (#3245) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-09 00:26:58 +05:30
Rushabh Shah c33767029e
HBASE-25860 Add metric for successful wal roll requests. (#3238) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-08 13:08:34 +05:30
Andrew Purtell 6978b8a0ef HBASE-25774 ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations 
Revert "HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2771)"

This reverts commit 5d7e2f71f7.
 2021-05-07 17:56:57 -07:00
Andrew Purtell 4de3841074 HBASE-25854 Remove redundant AM in-memory state changes in CatalogJanitor (#3234) 
In CatalogJanitor we schedule GCRegionProcedure to clean up both
filesystem and in-memory state after a split, and
GCMultipleMergedRegionsProcedure to do the same for merges. Both of these
procedures clean up in-memory state, but CatalogJanitor also does this
redundantly just after scheduling the procedures. The cleanup should be
done in only one place. Presumably we are using the procedures to do it in
a principled way. Remove the redundancy in CatalogJanitor and fix any
follow on issues, like test failures.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-06 09:17:35 -07:00
Andrew Purtell c32b6c8eff HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore (#3230) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-05 17:02:18 -07:00
Andrew Purtell 0fc6852f6a HBASE-25835 Ignore duplicate split requests from regionserver reports (#3218) 
Processing of the RS report happens asynchronously from other activities
which can mutate region state. For example, a split procedure may already
be running. A split procedure cannot succeed if the parent region is no
longer open, so we can ignore it in that case.

Note that submitting more than one split procedure for a given region is
harmless -- the split is fenced in the procedure handling -- but it would
be noisy in the logs. Only one procedure can succeed. The other
procedure(s) would abort during initialization and report failure with
WARN level logging.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
 2021-05-04 10:23:04 -07:00
Andrew Purtell fd3e28364c HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions (#3219) 
RegionStates#getAssignmentsForBalancer is used by the HMaster to
collect all regions of interest to the balancer for the next chore
iteration. We check if a table is in disabled state to exclude
regions that will not be of interest (because disabled regions are
or will be offline) or are in a state where they shouldn't be
mutated (like SPLITTING). The current checks are not actually
comprehensive.

Filter out regions not in OPEN or OPENING state when building the
set of interesting regions for the balancer to consider. Only
regions open (or opening) on the cluster are of interest to
balancing calculations for the current iteration. Regions in all
other states can be expected to not be of interest – either offline
(OFFLINE, or FAILED_*), not subject to balancer decisions now
(SPLITTING, SPLITTING_NEW, MERGING, MERGING_NEW), or will be
offline shortly (CLOSING) – until at least the next chore
iteration.

Add TRACE level logging.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-03 18:24:17 -07:00
Andrew Purtell 368d52fe99 HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip (#3223) 
We claim in a WARN level log line to be "Playing-it-safe skipping merge/
split gc'ing of regions from hbase:meta while regions-in-transition (RIT)"
but do not actually skip because of a missing return. Remove the warning.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-03 18:17:56 -07:00
Duo Zhang 08507b86a8 HBASE-25838 Use double instead of Double in StochasticLoadBalancer (#3221) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-04 09:10:35 +08:00
Andrew Purtell 8fd02a75f8 Preparing development version 2.4.4-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-04-27 14:58:08 -07:00
Andrew Purtell af14fd998c Preparing hbase release 2.4.3RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-04-27 14:58:03 -07:00
stack 12200e9d89 HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars (#3184) 
Need to add to allowed-licenses list too....

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-04-27 09:07:21 -07:00
Duo Zhang e765e19279 HBASE-25806 Backport the region location finder initialization fix in HBASE-25802 (#3196) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-26 13:07:41 +08:00
haxiaolin 1dc881ab85 HBASE-25754 StripeCompactionPolicy should support compacting cold regions (#3152) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-23 15:32:27 +08:00
Toshihiro Suzuki cc7c4352ee HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-04-22 14:14:36 +09:00
Duo Zhang a15fd5cff5 HBASE-25775 Use a special balancer to deal with maintenance mode (#3161) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-04-16 13:40:29 +08:00
Nick Dimiduk 7179c013bb HBASE-25770 Http InfoServers should honor gzip encoding when requested (#3159) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2021-04-15 13:53:01 -07:00
xiaoyu 22fd7d7f10
HBASE-25776 Use Class.asSubclass to fix the warning in StochasticLoadBalancer.loadCustomCostFunctions (#3163) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-15 23:37:05 +05:30
Duo Zhang 69e93c8137 HBASE-25767 CandidateGenerator.getRandomIterationOrder is too slow on large cluster (#3149) 
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-14 08:29:01 +08:00
DivyeshChandra 082ad7cf21 HBASE-25653 Add units and round off region size to 2 digits after decimal (#3046) 
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
 2021-04-13 09:07:19 -07:00
Duo Zhang d2cba187e5 HBASE-25759 The master services field in LocalityBasedCostFunction is never used (#3144) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-12 22:43:48 +08:00
Geoffrey Jacoby 843e1369f7 HBASE-25751 - Add writable TimeToPurgeDeletes to ScanOptions (#3137) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-09 13:09:28 -07:00
meiyi da6b1df643 HBASE-25747 Remove unused getWriteAvailable method in OperationQuota (#3133) 
Signed-off-by: stack <stack@apache.org>
 2021-04-09 10:26:26 +08:00