4aab6bb959
HBASE-27857 Fix timeout exception handling in HBaseClassTestRule. ( #5231 )
...
HBaseClassTestRule applies a timeout and a system exit rule to tests.
The timeout rule throws an exception if it hits the timeout threshold.
Since the timeout rule is applied after the system exit rule, the
system exit rule does not see the exception and does not re-enable
the system exit behavior which can cause maven to hang on some
tests.
This change applies the timeout rule before the system exit rule so
that normal system exit can be restored before the surefire forked
node is shutdown.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 220eacf7c5
2023-05-13 09:08:09 +08:00
9990d121b6
HBASE-27788 Skip family comparing when compare cells inner the store
2023-05-10 10:51:53 +08:00
1ae057b89f
HBASE-27536: Include more request information in slowlog for Scans ( #5166 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-04-16 10:11:35 -04:00
a67a8f7fd3
HBASE-27780 FileChangeWatcher improvements ( #5164 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-04-08 10:25:32 -04:00
5de1e4f88e
HBASE-27710 ByteBuff ref counting is too expensive for on-heap buffers ( #5115 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-03-17 15:24:28 -04:00
59434e9cd4
HBASE-27708 CPU hot-spot resolving User subject
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-03-16 13:57:02 +01:00
1ac6f1689f
HBASE-25709 Close region may stuck when region is compacting and skipped most cells read ( #4536 ) ( #5085 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-03-06 14:21:22 +08:00
3f25f33bc3
HBASE-27654 IndexBlockEncoding is missing in HFileContextBuilder copy constructor ( #5046 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-02-20 18:21:42 +08:00
5aab852a6d
Revert "HBASE-29654 IndexBlockEncoding is missing in HFileContextBuilder copy constructor ( #5041 )" ( #5045 )
...
This reverts commit f87c6b8505
2023-02-20 18:17:20 +08:00
f87c6b8505
HBASE-29654 IndexBlockEncoding is missing in HFileContextBuilder copy constructor ( #5041 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-02-20 17:59:45 +08:00
beab1eb208
HBASE-27648 CopyOnWriteArrayMap does not honor contract of ConcurrentMap.putIfAbsent ( #5031 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-02-17 19:18:19 -05:00
1fb311f22a
HBASE-27621 Also clear the Dictionary when resetting when reading compressed WAL file ( #5016 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 833b10e8ba
2023-02-11 19:54:57 +08:00
f619001dff
HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding ( #5006 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-02-04 12:20:17 -05:00
b0f2fc520f
HBASE-25516 [JDK17] reflective access Field.class.getDeclaredField("modifiers") not supported ( #3443 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
(cherry picked from commit 29cd782d25
2023-01-24 22:50:36 +08:00
655a4366a5
HBASE-27565 Make the initial corePoolSize configurable for ChoreService ( #4958 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-01-12 14:50:38 +01:00
e90a351d0d
HBASE-27347 Port FileWatcher from ZK to autodetect keystore/truststore changes in TLS connections (branch-2) ( #4897 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-11-28 08:06:29 -05:00
838638fdab
HBASE-27504 Remove duplicated config 'hbase.normalizer.merge.min_region_age.days' in hbase-default.xml ( #4894 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit c691091f78
2022-11-23 20:43:42 +08:00
ecf3debd42
HBASE-26913 Replication Observability Framework ( #4862 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-11-08 13:05:42 -08:00
738ed028ad
HBASE-27434 Use ${revision} as placeholder for maven version to make it easier to control the version from command line ( #4836 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 2fc879e863
2022-10-24 12:05:01 +08:00
5cab4be075
HBASE-27401 Clean up current broken 'n's in our javadoc ( #4812 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 63cdd026f0
2022-10-06 22:13:14 +08:00
f69b47275e
HBASE-27304 Support using IP to expose master/rs servers for some special scenarios ( #4798 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-25 22:50:06 +08:00
2a92819f0b
HBASE-27280 Add mutual authentication support to TLS ( #4796 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Andor Molnár <andor@cloudera.com>
2022-09-23 09:11:30 -04:00
9fb3aee22b
HBASE-27373 Fix new spotbugs warnings after upgrading spotbugs to 4.7.2 ( #4787 ) ( #4789 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-09-18 18:31:05 +08:00
bfd8501e23
HBASE-27314 Make index block be customized and configured ( #4763 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-08 09:27:53 +08:00
c3a30232a8
HBASE-27346 Autodetect key/truststore file type from file extension ( #4757 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-09-06 16:50:54 -04:00
9838c070aa
HBASE-27342 Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores ( #4751 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2022-09-01 10:06:15 +02:00
935b0d2277
HBASE-27321 The ReplicationLogCleaner is not thread safe but can be called from different threads at the same time ( #4730 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 37651ee1b0
2022-08-28 00:30:34 +08:00
c04edf7835
HBASE-20904 Prometheus /metrics http endpoint for monitoring ( #4691 )
...
Co-authored-by: Luca Kovacs <kovacs.luca.agota@gmail.com>
Co-authored-by: Madhusoodan P <akshayapataki123@gmail.com>
Co-authored-by: Luca Kovacs <lkovacs@cloudera.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f9ea7ee0d6
2022-08-24 11:31:16 +08:00
f509e43eaa
HBASE-27089 Add “commons.crypto.stream.buffer.size” configuration ( #4491 )
...
Co-authored-by: wenwj0 <wenweijian2@huawei.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 7ff927dbd8
2022-08-21 22:32:39 +08:00
f705e3e7d8
HBASE-27206 Clean up error-prone findings in hbase-common ( #4645 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fbe3b90e0c
2022-08-21 21:36:01 +08:00
aaad3a7077
HBASE-26982 Add index and bloom filter statistics of LruBlockCache on… ( #4376 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: stack <stack@apache.org>
2022-08-15 11:12:05 -07:00
b25ded026b
HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose ( #4695 ) ( #4701 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-15 09:16:29 -07:00
f36b34984f
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone ( #4617 )
...
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446 )
* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
* spotless:apply to fix HBaseTestingUtility
* Fix error-prone errors
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>
2022-08-12 14:29:26 -07:00
b820188c68
HBASE-27278 Improve TestTlsIPC to reuse existing IPC test code ( #4682 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 3309108ca7
2022-08-12 13:21:11 +08:00
2821cc518a
HBASE-26666 Add native TLS encryption support to RPC server/client ( #4666 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit f8dcf07055
2022-08-06 19:43:17 -04:00
bcec871959
HBASE-27153 Improvements to read-path tracing
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-08-02 14:09:39 +02:00
84e7cdb2c0
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone ( #4634 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061
2022-07-26 23:45:15 +08:00
99f2ab5aa8
HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 10:00:31 +08:00
d19805a7bc
HBASE-27195 Clean up netty worker/thread pool configuration ( #4619 )
...
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.
Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".
Also, native epoll is not limited to amd64. aarch64 supports it too.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
2022-07-18 17:42:37 -07:00
325f90bb40
HBASE-27144 Add special rpc handlers for bulkload operations ( #4558 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit ff8eb59709
2022-07-17 21:41:27 +08:00
e7551c1359
HBASE-26950 Use AsyncConnection in ReplicationSink ( #4595 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2022-07-09 10:25:17 +08:00
57a3781160
HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse ( #4592 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-04 16:37:29 -04:00
0727015bf5
HBASE-22749 Distributed MOB compactions ( #4581 )
...
* HBASE-22749 Distributed MOB compactions
- MOB compaction is now handled in-line with per-region compaction on region
servers
- regions with mob data store per-hfile metadata about which mob hfiles are
referenced
- admin requested major compaction will also rewrite MOB files; periodic RS
initiated major compaction will not
- periodically a chore in the master will initiate a major compaction that
will rewrite MOB values to ensure it happens. controlled by
'hbase.mob.compaction.chore.period'. default is weekly
- control how many RS the chore requests major compaction on in parallel
with 'hbase.mob.major.compaction.region.batch.size'. default is as
parallel as possible.
- periodic chore in master will scan backing hfiles from regions to get the
set of referenced mob hfiles and archive those that are no longer
referenced. control period with 'hbase.master.mob.cleaner.period'
- Optionally, RS that are compacting mob files can limit write
amplification by not rewriting values from mob hfiles over a certain size
limit. opt-in by setting 'hbase.mob.compaction.type' to 'optimized'.
control threshold by 'hbase.mob.compactions.max.file.size'.
default is 1GiB
- Should smoothly integrate with existing MOB users via rolling upgrade.
will delay old MOB file cleanup until per-region compaction has managed
to compact each region at least once so that used mob hfile metadata can
be gathered.
* HBASE-22749 Distributed MOB compactions
fix RestrictedApi
Co-authored-by: Vladimir Rodionov <vrodionov@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-06-30 20:44:45 +01:00
a9f9701b72
Revert "HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted"
...
This reverts commit 96693e8b40
2022-06-25 10:57:47 -07:00
e8f04a8a7f
HBASE-27146 Avoid CellUtil.cloneRow in MetaCellComparator ( #4571 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: SiCheng-Zheng <643463623@qq.com>
(cherry picked from commit b1691a5318
2022-06-24 22:37:27 +08:00
da76941b0e
HBASE-27111 Make Netty channel bytebuf allocator configurable. ( #4525 )
...
Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
This allows the user to add a custom implementation, perhaps for debugging.
Also updates ReflectionUtils with a new helper method.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestNettyRpcServer.java
2022-06-23 15:18:32 -07:00
96693e8b40
HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
...
Created OnheapDecodedCell and OffheapDecodedExtendedCell objects with duplicate copy of
ByteBuffer's underlying array instead of original ByteBuffer
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2022-06-13 14:21:03 +02:00
18a1a4bdac
HBASE-27097 SimpleRpcServer is broken ( #4521 )
...
Apply https://issues.apache.org/jira/secure/attachment/13044874/MultiByteBuff.patch
from Lijin Bin on JIRA.
Co-authored-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-11 10:41:38 -07:00
9b8bb580b9
HBASE-26366 Provide meaningful parent spans to ZK interactions
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-10 16:26:37 +02:00
50e2ef24d5
HBASE-27038 CellComparator should extend Serializable ( #4492 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-06 17:45:20 -04:00
Jonathan Albrecht
bsglz
Ray Mattingly
Bryan Beaudreault
Nick Dimiduk
Xiaolin Ha
chenglei
Duo Zhang
Wei-Chiu Chuang
Peter Somogyi
Andor Molnár
mokai
Rushabh Shah
LiangJun He
Bryan Beaudreault
binlijin
Luca Kovács
wenwj0
Andrew Purtell
liangxs
huaxiangsun
SiCheng-Zheng
Bryan Beaudreault
BukrosSzabolcs
Mohammad Arshad