fc93fbac9a
HBASE-27129 Add a config that allows us to configure region-level storage policies ( #4547 )
...
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f238a92b6d
2022-08-22 10:07:27 +08:00
f705e3e7d8
HBASE-27206 Clean up error-prone findings in hbase-common ( #4645 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fbe3b90e0c
2022-08-21 21:36:01 +08:00
8a10dbf6c4
HBASE-27268 In trace log mode, the client does not print callId/startTime and the server does not print receiveTime ( #4710 )
...
Co-authored-by: jay.zhu <jay.zhu@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 32c135d8a5
2022-08-20 21:09:43 +08:00
d073404706
HBASE-26969:Eliminate MOB renames when SFT is enabled
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
closes #4712
2022-08-19 11:33:48 +02:00
1176b484d1
HBASE-25970 MOB data loss - incorrect concatenation of MOB_FILE_REFS ( #3355 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2022-08-19 11:20:53 +02:00
1986a6ec31
HBASE-24163 MOB compactor implementations should use format specifiers when calling String.format
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-08-17 13:46:18 +02:00
c2dff398d3
HBASE-27279 Make SslHandler work with SaslWrapHandler/SaslUnwrapHandler ( #4705 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: Andor Molnár <andor@cloudera.com>
(cherry picked from commit 2b9d36869f
2022-08-16 21:37:47 +08:00
3acf920e1f
HBASE-27264 Add options to consider compressed size when delimiting blocks during hfile writes ( #4675 )
...
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
2022-08-15 22:36:44 +01:00
aaad3a7077
HBASE-26982 Add index and bloom filter statistics of LruBlockCache on… ( #4376 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: stack <stack@apache.org>
2022-08-15 11:12:05 -07:00
b25ded026b
HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose ( #4695 ) ( #4701 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-15 09:16:29 -07:00
f36b34984f
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone ( #4617 )
...
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446 )
* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
* spotless:apply to fix HBaseTestingUtility
* Fix error-prone errors
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>
2022-08-12 14:29:26 -07:00
b820188c68
HBASE-27278 Improve TestTlsIPC to reuse existing IPC test code ( #4682 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 3309108ca7
2022-08-12 13:21:11 +08:00
25e192b807
HBASE-27248 WALPrettyPrinter add print timestamp ( #4659 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6b2e9d14a7
2022-08-10 21:30:55 +08:00
e99faad83e
HBASE-27241 Add metrics for evaluating cost and effectiveness of bloom filters ( #4669 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-09 15:25:47 -04:00
beb7c6d490
HBASE-27283 Use readTO instead of hard coded RpcClient.DEFAULT_SOCKET_TIMEOUT_READ when creating ReadTimeoutHandler in NettyRpcConnection ( #4685 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 5919b30b6d
2022-08-09 16:22:57 +08:00
30f6a17056
HBASE-27282 CME in AuthManager causes region server crash ( #4684 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit d6641626a9
2022-08-09 09:56:26 +08:00
a85de838ab
HBASE-27265 : Tool to read StoreFileTrackerFile ( #4673 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-08 15:31:14 +01:00
c81517638c
HBASE-27270 Add bulkloadExecutor.resizeQueues ( #4677 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4ebf71982c
2022-08-08 18:15:25 +08:00
2821cc518a
HBASE-26666 Add native TLS encryption support to RPC server/client ( #4666 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit f8dcf07055
2022-08-06 19:43:17 -04:00
2e7a983923
HBASE-27273 Should stop autoRead and skip all the bytes when rpc request too big ( #4679 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 486d19e99f
2022-08-06 20:31:27 +08:00
c5b22d973b
HBASE-27269 The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect ( #4678 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 2dc26082be
2022-08-04 23:14:24 +08:00
bcec871959
HBASE-27153 Improvements to read-path tracing
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-08-02 14:09:39 +02:00
dbad3d1175
HBASE-27257 Remove unnecessary usage of CachedBlocksByFile from RS UI ( #4667 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-07-29 15:28:54 -04:00
5f231d3fb3
HBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler ( #4624 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 0c4263a18b
2022-07-27 09:10:32 +08:00
4c08a8a423
HBASE-27087 TestQuotaThrottle times out
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-26 14:10:49 -07:00
827c1ab5dd
HBASE-27229 BucketCache statistics should not count evictions by hfile ( #4639 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-26 17:00:46 -04:00
84e7cdb2c0
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone ( #4634 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061
2022-07-26 23:45:15 +08:00
1fd772c3b1
HBASE-27225 Add BucketAllocator bucket size statistic logging ( #4637 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-07-25 20:56:05 -04:00
80c33e0d46
HBASE-27232 Fix checking for encoded block size when deciding if bloc… ( #4640 )
...
Signed-off-by: Andor Molnár <andor@cloudera.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit d5ed8f59e5
2022-07-25 12:08:09 +01:00
db98fab114
HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky ( #4636 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 31fc97edee
2022-07-21 14:52:39 +08:00
468715bff3
Revert "HBASE-27152 Under compaction mark may leak ( #4568 )"
...
This reverts commit 5c5ceb1ef2
2022-07-21 09:04:17 +08:00
5c5ceb1ef2
HBASE-27152 Under compaction mark may leak ( #4568 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit da27a67a1e
2022-07-20 08:51:58 +08:00
eb73492eac
HBASE-20499 Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
(cherry picked from commit 6d080762ef
2022-07-19 13:48:07 +08:00
642e933cda
HBASE-27144 Addendum fix spotbug warning
2022-07-19 13:41:07 +08:00
99f2ab5aa8
HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 10:00:31 +08:00
d19805a7bc
HBASE-27195 Clean up netty worker/thread pool configuration ( #4619 )
...
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.
Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".
Also, native epoll is not limited to amd64. aarch64 supports it too.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
2022-07-18 17:42:37 -07:00
f76c0d9021
HBASE-27202 Clean up error-prone findings in hbase-balancer ( #4623 )
...
On branch-2, the affected code is in hbase-server.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/RegionLocationFinder.java
2022-07-18 16:00:26 -07:00
824785eeeb
HBASE-27211 Data race in MonitoredTaskImpl could cause split wal failure ( #4630 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 9ab0b1504f
2022-07-18 20:08:38 +08:00
325f90bb40
HBASE-27144 Add special rpc handlers for bulkload operations ( #4558 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit ff8eb59709
2022-07-17 21:41:27 +08:00
ca15d39026
HBASE-27053 IOException during caching of uncompressed block to the block cache ( #4610 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: wenwj0 <wenweijian2@huawei.com>
2022-07-16 17:33:27 -04:00
8d3693a018
HBASE-27192 The retry number for TestSeparateClientZKCluster is too small ( #4614 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 648bb6d0eb
2022-07-14 08:52:12 +08:00
70a561eb63
HBASE-27194 Add test coverage for SimpleRpcServer ( #4616 )
...
Add test coverage for SimpleRpcServer.
Improve the way we test both SimpleRpcServer and NettyRpcServer. Use
LoadTestKVGenerator to generate random values with varying sizes between
1000 bytes and 1M bytes, and also to verify them when reading the values
back.
Add secure test coverage for both SimpleRpcServer and NettyRpcServer.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-13 11:22:29 -07:00
47230c4727
HBASE-27193 TestZooKeeper is flaky ( #4615 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit 2662607b71
2022-07-13 10:34:15 +08:00
fbfe7b3880
HBASE-27161 Improve TestMultiRespectsLimits ( #4586 )
...
TestMultiRespectLimits submits a put with an empty value. This appears
to not be what was intended. Instead, use a single byte.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-07-12 11:20:56 -07:00
0018cbec58
HBASE-27097 SimpleRpcServer is broken ( #4613 )
...
Replace BufferChain#write(channel,int) with a simpler #write(channel)
implementation that does not attempt to "chunk" data to be written. This
method was used exclusively by SimpleRpcServer. The code was unnecessarily
complex and caused short writes when values were large, so was corrected
and simplified. Any difference in performance from this change will be
limited to SimpleRpcServer. Testing under load confirms the fix and does
not show significant regression.
SimpleRpcServer and its related code is now also marked as @Deprecated.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
2022-07-12 11:09:05 -07:00
b2a145661f
HBASE-27149 Server should close scanner if client times out before results are ready ( #4604 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-11 22:36:42 -04:00
98ec7586b4
HBASE-27188 Report maxStoreFileCount in jmx ( #4609 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-11 22:14:01 -04:00
9ccb0f96b9
HBASE-27186 Report block cache size metrics separately for L1 and L2 ( #4608 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-11 22:01:06 -04:00
68f539ef94
HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed ( #4611 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5bc8670322
2022-07-12 09:44:18 +08:00
e7551c1359
HBASE-26950 Use AsyncConnection in ReplicationSink ( #4595 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2022-07-09 10:25:17 +08:00
tianhang
Andrew Purtell
Fantasy-Jay
BukrosSzabolcs
Peter Somogyi
Sean Busbey
Duo Zhang
Wellington Ramos Chevreuil
liangxs
huaxiangsun
SiCheng-Zheng
Bryan Beaudreault
chaijunjie0101
Abhradeep Kundu
Andor Molnár
LiangJun He
Nick Dimiduk
Xiaolin Ha
Nihal Jain
chenglei