Commit Graph

18164 Commits

Author SHA1 Message Date
Bryan Beaudreault a8a67d4314 HBASE-27186 Report block cache size metrics separately for L1 and L2 (#4608)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-11 22:09:57 -04:00
Duo Zhang d054c93be1 HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed (#4611)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5bc8670322)
2022-07-12 09:47:03 +08:00
Norman Maurer ff710b8252 HBASE-27180 Fix multiple possible buffer leaks (#4597)
* Fix multiple possible buffer leaks

Motivation:

When using ByteBuf you need to be very careful about releasing it as otherwise you might leak data. There were various places in the code-base where such a leak could happen.

Modifications:

- Fix possible buffer leaks
- Ensure we call touch(...) so its easier to debug buffer leaks

Result:

Fix buffer leaks

* Formatting

* Revert some changes as requested

* revert touch

* Also release checksum and header buffers

Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 2197b3806b)
2022-07-08 08:32:33 +08:00
Duo Zhang decb96edd2 HBASE-26708 Netty leak detected and OutOfDirectMemoryError due to direct memory buffering with SASL implementation (#4596)
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 816e919e95)
2022-07-07 16:04:40 +08:00
Ujjawal 60521caf9f
HBASE-27175 - Failure to cleanup WAL split dir log should be at INFO level (#4593)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-06 16:35:22 -07:00
Bryan Beaudreault 87f22813f4 HBASE-27048 Server side scanner time limit should account for time in queue (#4562)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-06 14:21:56 -04:00
tianhang c725dc98e5 HBASE-27171 Fix Annotation Error in HRegionFileSystem (#4588)
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6031a3a8d4)
2022-07-06 12:39:35 +08:00
Duo Zhang 2cd30e5f00 HBASE-23330: Fix delegation token fetch with MasterRegistry (#1084) (#4598)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit d8b3f55518)

Co-authored-by: Bharath Vissapragada <bharathv@apache.org>
2022-07-06 12:00:09 +08:00
Bryan Beaudreault f36b8eadd8 HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse (#4592)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-04 16:47:47 -04:00
Duo Zhang ab722fe765 HBASE-27169 TestSeparateClientZKCluster is flaky (#4587)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 03c23beb57)
2022-07-01 22:16:52 +08:00
Ishika ba9aa5df99 HBASE-26218 Add logs in Canary tool (#4501)
Co-authored-by: Ishika Soni <isoni@isoni-ltmag9r.internal.salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit acfbc3ff7d)
2022-06-30 23:28:41 +08:00
Duo Zhang aaf6435317 HBASE-27160 ClientZKSyncer.deleteDataForClientZkUntilSuccess should break from the loop when deletion is succeeded (#4579)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit d7f6861e8d)
2022-06-26 15:29:47 +08:00
Bryan Beaudreault 9735b545d1 HBASE-27060 Addendum spotless fix (#4580)
Signed-off-by: Sean Busbey <busbey@apache.org>
2022-06-25 17:36:49 -04:00
Bryan Beaudreault 4c24c5d8e0 HBASE-27060 Addendum fix HBaseTestingUtility import in test 2022-06-25 01:49:24 -04:00
Bryan Beaudreault 4e4b3baed4 HBASE-27060 Allow sharing connections between AggregationClient instances (#4566)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-24 10:51:10 -04:00
Duo Zhang 2472a77b3d HBASE-27146 Avoid CellUtil.cloneRow in MetaCellComparator (#4571)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: SiCheng-Zheng <643463623@qq.com>
(cherry picked from commit b1691a5318)
2022-06-24 22:37:46 +08:00
Bryan Beaudreault ce3a37ebac HBASE-26790 getAllRegionLocations can cache locations with null hostname (#4575)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-24 07:47:37 -04:00
Bryan Beaudreault 207b7d0fe5 HBASE-26945 Quotas causes too much load on meta for large clusters (#4576)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-06-23 16:48:52 -04:00
Andrew Purtell 54be59642d Preparing development version 2.4.14-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-22 20:15:07 -07:00
Andrew Purtell 90fb1ddc1d Preparing hbase release 2.4.13RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-22 20:15:03 -07:00
Andrew Purtell 847665d9d6 HBASE-27151 TestMultiRespectsLimits.testBlockMultiLimits repeatable failure
Revert "HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted"

This reverts commit b2f71f2049.
2022-06-22 20:11:42 -07:00
Andrew Purtell 21481143d8 HBASE-27050 Support unit test pattern matching again (#4447)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-22 19:46:46 -07:00
Andrew Purtell 67592eaba0 Preparing development version 2.4.14-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-22 09:51:14 -07:00
Andrew Purtell 249b5f1170 Preparing hbase release 2.4.13RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-22 09:51:11 -07:00
Duo Zhang ba3d86c972 HBASE-27141 Upgrade hbase-thirdparty dependency to 4.1.1 (#4552)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
(cherry picked from commit 9bae50f83c)

Conflicts:
	pom.xml
2022-06-22 23:38:41 +08:00
Andrew Purtell cd2a7ff1c0 HBASE-27051 TestReplicationSource.testReplicationSourceInitializingMetric is flaky (#4448)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-22 14:05:52 +02:00
Tak Lon (Stephen) Wu 230e0a24ec HBASE-27143 Add hbase-unsafe as a dependency for a MR job triggered by hbase shell (#4554)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-21 13:42:21 -07:00
xiaozhang0319 745cc7bc58 HBASE-27099 The log printing fspread/fsread cost time unit should be milliseconds (#4500)
Co-authored-by: xiaozhang <issac.zhang@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b498efdbc1)
2022-06-21 00:14:44 +08:00
SiCheng-Zheng eb41b47baa HBASE-27128 when open archiveRetries totalLogSize calculation mistake (#4546)
Co-authored-by: zhengsicheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5833756e5e)
2022-06-20 23:48:51 +08:00
Xiaolin Ha 51e8f6e6dc
HBASE-27125 The batch size of cleaning expired mob files should have an upper bound(addendum) (#4553)
fix ut error
2022-06-20 11:44:31 +08:00
Xiaolin Ha 15a88efb17 HBASE-27125 The batch size of cleaning expired mob files should have an upper bound (#4541)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-18 02:09:00 +08:00
litao e21f164050
HBASE-27117 Update the method comments for RegionServerAccounting (#4532) 2022-06-16 17:20:19 -07:00
Mohammad Arshad b2f71f2049 HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
Created OnheapDecodedCell and OffheapDecodedExtendedCell objects with duplicate copy of
ByteBuffer's underlying array instead of original ByteBuffer

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2022-06-13 14:21:38 +02:00
Viraj Jasani ae1a0ed94e Revert "HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#3117)" (#4524)
This reverts commit f3a48d1910.

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
2022-06-12 10:25:28 -07:00
Andrew Purtell fdb91176b5 HBASE-27097 SimpleRpcServer is broken (#4521)
Apply https://issues.apache.org/jira/secure/attachment/13044874/MultiByteBuff.patch
from Lijin Bin on JIRA.

Co-authored-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-11 10:46:24 -07:00
Duo Zhang 42a51545f1 Revert "HBASE-27084 Add spotless:check in mvn verify stage (#4482)"
This reverts commit bcea930b23.
2022-06-12 00:13:28 +08:00
wenwj0 824b7295e8 HBASE-26923 PerformanceEvaluation support encryption option (#4489)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b7065c1cf5)

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
2022-06-11 21:30:34 +08:00
Nick Dimiduk c8f3c384ae HBASE-27102 Vacate the .idea folder in order to simplify spotless configuration
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-10 15:58:42 +02:00
Nick Dimiduk a3153bfde5
HBASE-27095 HbckChore should produce a report
In #4470 for HBASE-26192, it was noted that the HbckChore is kind of a pain to use and test
because it maintains a bunch of local state. By contract, the CatalogJanitorChore makes a nice
self-contained report. Let's update HbckChore to do the same.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-10 15:47:23 +02:00
wangzhi 3d82d2d9e7 HBASE-27093 AsyncNonMetaRegionLocator:put Complete CompletableFuture outside lock block (#4496)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 176c43c5ad)
2022-06-07 12:18:54 +08:00
Bryan Beaudreault f51e24773f HBASE-27038 CellComparator should extend Serializable (#4492)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-06 17:48:39 -04:00
Duo Zhang bcea930b23 HBASE-27084 Add spotless:check in mvn verify stage (#4482)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit a060e71c53)

Conflicts:
	pom.xml
2022-06-06 23:49:26 +08:00
liuzhuang2017 999801adad HBASE-27080 Optimize debug output log of ConstantSizeRegionSplitPolicy class. (#4481)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 12a9e6e5ec)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
2022-06-06 13:43:58 +08:00
huaxiangsun 976dbe8b2b
HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4442) (#4485)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-03 15:28:19 -07:00
skysiders 2e08c69c5c
HBASE-26985 check permission for SecureBulkLoadManager (#4379)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-02 15:30:31 -07:00
Duo Zhang 9ea3e39d7a HBASE-27046 The filenum in AbstractFSWAL should be monotone increasing (#4449)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 313a37f7ac)
2022-06-03 00:23:45 +08:00
Xiaolin Ha df9739979e HBASE-26680 Close and do not write trailer for the broken WAL writer(addendum) (#4405)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 80f410ec3f)
2022-06-03 00:08:53 +08:00
LiangJun He 3ce6e1afe7 HBASE-27039 Some methods of MasterRegion should be annotated for testing only (#4433)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 15002fccb4)
2022-06-02 21:39:20 +08:00
Duo Zhang 2d70427a32 HBASE-27023 Fix license issues after running spotless:apply (#4458)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit e555ac4a99)
2022-06-02 20:25:35 +08:00
Andrew Purtell f1c7f3a2f0 HBASE-27079 Lower some DEBUG level logs in ReplicationSourceWALReader to TRACE (#4476)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-05-31 12:43:50 -07:00