Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,373 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

9245 Commits

Author SHA1 Message Date
SiCheng-Zheng 4ebf71982c
HBASE-27270 Add bulkloadExecutor.resizeQueues (#4677) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-08 18:14:49 +08:00
Andor Molnár f8dcf07055
HBASE-26666 Add native TLS encryption support to RPC server/client (#4666) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2022-08-06 19:09:54 -04:00
Duo Zhang 486d19e99f
HBASE-27273 Should stop autoRead and skip all the bytes when rpc request too big (#4679) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-08-06 20:08:07 +08:00
LiangJun He 2dc26082be
HBASE-27269 The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect (#4678) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-04 22:42:36 +08:00
Nick Dimiduk d734acc00e HBASE-27153 Improvements to read-path tracing 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-08-02 14:09:09 +02:00
Bryan Beaudreault e8c14ee308
HBASE-27257 Remove unnecessary usage of CachedBlocksByFile from RS UI (#4667) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-07-29 15:28:28 -04:00
Duo Zhang 0c4263a18b
HBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler (#4624) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-27 09:00:42 +08:00
Andrew Purtell 9ba868d2ad HBASE-27087 TestQuotaThrottle times out 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 14:09:48 -07:00
Bryan Beaudreault 277058e18d
HBASE-27229 BucketCache statistics should not count evictions by hfile (#4639) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 16:59:57 -04:00
Duo Zhang 8b091c4061
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 23:42:37 +08:00
Bryan Beaudreault ebd189157e
HBASE-27225 Add BucketAllocator bucket size statistic logging (#4637) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-07-25 20:54:40 -04:00
Wellington Ramos Chevreuil d5ed8f59e5
HBASE-27232 Fix checking for encoded block size when deciding if bloc… (#4640) 
Signed-off-by: Andor Molnár <andor@cloudera.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-25 11:17:16 +01:00
chenglei 0ae42ddaee
HBASE-27230 RegionServer should be aborted when WAL.sync throws Timeo… (#4641) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-24 15:08:03 +08:00
Duo Zhang 31fc97edee
HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky (#4636) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-21 14:26:52 +08:00
Duo Zhang 75a05b5cd0 Revert "HBASE-27152 Under compaction mark may leak (#4568)" 
This reverts commit da27a67a1e.
 2022-07-21 09:02:40 +08:00
Xiaolin Ha da27a67a1e
HBASE-27152 Under compaction mark may leak (#4568) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-20 08:49:10 +08:00
Duo Zhang acf144717b HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 10:00:43 +08:00
Andrew Purtell 745142adb1
HBASE-27195 Clean up netty worker/thread pool configuration (#4619) 
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.

Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".

Also, native epoll is not limited to amd64. aarch64 supports it too.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-18 15:48:40 -07:00
Duo Zhang 9ab0b1504f
HBASE-27211 Data race in MonitoredTaskImpl could cause split wal failure (#4630) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-18 19:25:06 +08:00
SiCheng-Zheng ff8eb59709
HBASE-27144 Add special rpc handlers for bulkload operations (#4558) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-17 21:21:58 +08:00
Bryan Beaudreault 70a2ee1716
HBASE-27053 IOException during caching of uncompressed block to the block cache (#4610) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: wenwj0 <wenweijian2@huawei.com>
 2022-07-16 17:30:55 -04:00
Andrew Purtell 01b45e285d Amend HBASE-27194 Add test coverage for SimpleRpcServer (#4616) 
When backporting to branch-2 it was discovered that the order of tear
down operations requires the KDC to be torn down first, although on
master branch the original code did not produce the same errors (a NPE
in KDC#stop). Forward port the difference for maintainability.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-13 11:23:07 -07:00
Andrew Purtell 84f0d145d7
HBASE-27194 Add test coverage for SimpleRpcServer (#4616) 
Add test coverage for SimpleRpcServer.

Improve the way we test both SimpleRpcServer and NettyRpcServer. Use
LoadTestKVGenerator to generate random values with varying sizes between
1000 bytes and 1M bytes, and also to verify them when reading the values
back.

Add secure test coverage for both SimpleRpcServer and NettyRpcServer.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-13 09:28:30 -07:00
Duo Zhang 2662607b71
HBASE-27193 TestZooKeeper is flaky (#4615) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2022-07-13 10:32:01 +08:00
Andrew Purtell 7cb7100de5
HBASE-27161 Improve TestMultiRespectsLimits (#4586) 
TestMultiRespectLimits submits a put with an empty value. This appears
to not be what was intended. Instead, use a single byte.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-12 11:19:35 -07:00
Andrew Purtell ed32043da0
HBASE-27097 SimpleRpcServer is broken (#4613) 
Replace BufferChain#write(channel,int) with a simpler #write(channel)
implementation that does not attempt to "chunk" data to be written. This
method was used exclusively by SimpleRpcServer. The code was unnecessarily
complex and caused short writes when values were large, so was corrected
and simplified. Any difference in performance from this change will be
limited to SimpleRpcServer. Testing under load confirms the fix and does
not show significant regression.

SimpleRpcServer and its related code is now also marked as @Deprecated.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-12 11:07:23 -07:00
Duo Zhang 648bb6d0eb
HBASE-27192 The retry number for TestSeparateClientZKCluster is too small (#4614) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-07-12 19:08:41 +08:00
Bryan Beaudreault 886e31952f
HBASE-27149 Server should close scanner if client times out before results are ready (#4604) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 22:22:00 -04:00
Bryan Beaudreault 6e66e78177
HBASE-27188 Report maxStoreFileCount in jmx (#4609) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 22:11:58 -04:00
Bryan Beaudreault 779c4e2316
HBASE-27186 Report block cache size metrics separately for L1 and L2 (#4608) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 22:00:04 -04:00
Duo Zhang 5bc8670322
HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed (#4611) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-12 09:40:19 +08:00
Bryan Beaudreault 6886d02a91
HBASE-27078 Allow configuring a separate timeout for meta scans (#4557) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-07 10:02:14 -04:00
Duo Zhang 816e919e95
HBASE-26708 Netty leak detected and OutOfDirectMemoryError due to direct memory buffering with SASL implementation (#4596) 
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-07 15:55:26 +08:00
Duo Zhang 1643c45770 HBASE-27157 Addendum fix compile error after revert 2022-07-07 10:26:13 +08:00
Duo Zhang 0116aff9c0 Revert "HBASE-27157 Potential race condition in WorkerAssigner (#4577)" 
This reverts commit f76d8554ca.
 2022-07-07 08:36:21 +08:00
Ujjawal a31cc931fd
HBASE-27175 - Failure to cleanup WAL split dir log should be at INFO level (#4593) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-06 16:32:52 -07:00
Bryan Beaudreault b9706916cc
HBASE-27048 Server side scanner time limit should account for time in queue (#4562) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-06 11:57:37 -04:00
Duo Zhang 6f4a0476f2 HBASE-27148 Addendum uncomment the shutdown code in TestVerifyReplicationCrossDiffHdfs 2022-07-06 14:48:08 +08:00
Duo Zhang 41972cb460
HBASE-27148 Move minimum hadoop 3 support version to 3.2.3 (#4561) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-06 12:49:47 +08:00
Ruanhui f76d8554ca
HBASE-27157 Potential race condition in WorkerAssigner (#4577) 
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
 2022-07-06 10:59:13 +08:00
tianhang 6031a3a8d4
HBASE-27171 Fix Annotation Error in HRegionFileSystem (#4588) 
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-06 10:58:34 +08:00
Bryan Beaudreault f3f292fad4
HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse (#4592) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-04 16:18:45 -04:00
Duo Zhang 03c23beb57
HBASE-27169 TestSeparateClientZKCluster is flaky (#4587) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-01 22:09:01 +08:00
Ishika acfbc3ff7d
HBASE-26218 Add logs in Canary tool (#4501) 
Co-authored-by: Ishika Soni <isoni@isoni-ltmag9r.internal.salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-30 21:53:20 +08:00
Duo Zhang 9bcb7493e0
HBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master (#4582) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-06-29 11:04:21 +08:00
Ruanhui 9c2bc55350
HBASE-27158 Add namespace column family to UNDELETABLE_META_COLUMNFAMILIES (#4578) 
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-29 10:27:53 +08:00
Andrew Purtell ca34bdccc3
HBASE-27166 WAL value compression minor improvements (#4584) 
A larger IO buffer for absorbing WALCodec writes can improve the compression
ratio of larger values, because the compressor will be given a larger internal
buffer over which there will be more match opportunities. Does not impact the
ability to read existing written files.

Also, reset the BAOS internal buffer on the way out of compress() so potential
large-ish buffers do not linger on the heap longer than necessary.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-28 12:49:25 -07:00
Duo Zhang d7f6861e8d
HBASE-27160 ClientZKSyncer.deleteDataForClientZkUntilSuccess should break from the loop when deletion is succeeded (#4579) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2022-06-26 15:22:25 +08:00
Andrew Purtell 4f88a3c4bb
HBASE-27111 Make Netty channel bytebuf allocator configurable. (#4525) 
Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
   preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
  name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
  This allows the user to add a custom implementation, perhaps for debugging.

Also updates ReflectionUtils with a new helper method.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-06-23 15:13:29 -07:00
Kerasone b8558d30d0
HBASE-27001 The deleted variable cannot be printed out (#4479) 
* The deleted variable cannot be printed out, add it

Co-authored-by: selina.yan <selina.yan@huolala.cn>


Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2022-06-23 22:41:32 +05:30