1fb311f22a
HBASE-27621 Also clear the Dictionary when resetting when reading compressed WAL file ( #5016 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 833b10e8ba
2023-02-11 19:54:57 +08:00
f619001dff
HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding ( #5006 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-02-04 12:20:17 -05:00
b0f2fc520f
HBASE-25516 [JDK17] reflective access Field.class.getDeclaredField("modifiers") not supported ( #3443 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
(cherry picked from commit 29cd782d25
2023-01-24 22:50:36 +08:00
655a4366a5
HBASE-27565 Make the initial corePoolSize configurable for ChoreService ( #4958 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-01-12 14:50:38 +01:00
e90a351d0d
HBASE-27347 Port FileWatcher from ZK to autodetect keystore/truststore changes in TLS connections (branch-2) ( #4897 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-11-28 08:06:29 -05:00
838638fdab
HBASE-27504 Remove duplicated config 'hbase.normalizer.merge.min_region_age.days' in hbase-default.xml ( #4894 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit c691091f78
2022-11-23 20:43:42 +08:00
ecf3debd42
HBASE-26913 Replication Observability Framework ( #4862 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-11-08 13:05:42 -08:00
738ed028ad
HBASE-27434 Use ${revision} as placeholder for maven version to make it easier to control the version from command line ( #4836 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 2fc879e863
2022-10-24 12:05:01 +08:00
5cab4be075
HBASE-27401 Clean up current broken 'n's in our javadoc ( #4812 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 63cdd026f0
2022-10-06 22:13:14 +08:00
f69b47275e
HBASE-27304 Support using IP to expose master/rs servers for some special scenarios ( #4798 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-25 22:50:06 +08:00
2a92819f0b
HBASE-27280 Add mutual authentication support to TLS ( #4796 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Andor Molnár <andor@cloudera.com>
2022-09-23 09:11:30 -04:00
9fb3aee22b
HBASE-27373 Fix new spotbugs warnings after upgrading spotbugs to 4.7.2 ( #4787 ) ( #4789 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-09-18 18:31:05 +08:00
bfd8501e23
HBASE-27314 Make index block be customized and configured ( #4763 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-08 09:27:53 +08:00
c3a30232a8
HBASE-27346 Autodetect key/truststore file type from file extension ( #4757 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-09-06 16:50:54 -04:00
9838c070aa
HBASE-27342 Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores ( #4751 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2022-09-01 10:06:15 +02:00
935b0d2277
HBASE-27321 The ReplicationLogCleaner is not thread safe but can be called from different threads at the same time ( #4730 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 37651ee1b0
2022-08-28 00:30:34 +08:00
c04edf7835
HBASE-20904 Prometheus /metrics http endpoint for monitoring ( #4691 )
...
Co-authored-by: Luca Kovacs <kovacs.luca.agota@gmail.com>
Co-authored-by: Madhusoodan P <akshayapataki123@gmail.com>
Co-authored-by: Luca Kovacs <lkovacs@cloudera.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f9ea7ee0d6
2022-08-24 11:31:16 +08:00
f509e43eaa
HBASE-27089 Add “commons.crypto.stream.buffer.size” configuration ( #4491 )
...
Co-authored-by: wenwj0 <wenweijian2@huawei.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 7ff927dbd8
2022-08-21 22:32:39 +08:00
f705e3e7d8
HBASE-27206 Clean up error-prone findings in hbase-common ( #4645 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fbe3b90e0c
2022-08-21 21:36:01 +08:00
aaad3a7077
HBASE-26982 Add index and bloom filter statistics of LruBlockCache on… ( #4376 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: stack <stack@apache.org>
2022-08-15 11:12:05 -07:00
b25ded026b
HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose ( #4695 ) ( #4701 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-15 09:16:29 -07:00
f36b34984f
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone ( #4617 )
...
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446 )
* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
* spotless:apply to fix HBaseTestingUtility
* Fix error-prone errors
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>
2022-08-12 14:29:26 -07:00
b820188c68
HBASE-27278 Improve TestTlsIPC to reuse existing IPC test code ( #4682 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 3309108ca7
2022-08-12 13:21:11 +08:00
2821cc518a
HBASE-26666 Add native TLS encryption support to RPC server/client ( #4666 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit f8dcf07055
2022-08-06 19:43:17 -04:00
bcec871959
HBASE-27153 Improvements to read-path tracing
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-08-02 14:09:39 +02:00
84e7cdb2c0
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone ( #4634 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061
2022-07-26 23:45:15 +08:00
99f2ab5aa8
HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 10:00:31 +08:00
d19805a7bc
HBASE-27195 Clean up netty worker/thread pool configuration ( #4619 )
...
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.
Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".
Also, native epoll is not limited to amd64. aarch64 supports it too.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
2022-07-18 17:42:37 -07:00
325f90bb40
HBASE-27144 Add special rpc handlers for bulkload operations ( #4558 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit ff8eb59709
2022-07-17 21:41:27 +08:00
e7551c1359
HBASE-26950 Use AsyncConnection in ReplicationSink ( #4595 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2022-07-09 10:25:17 +08:00
57a3781160
HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse ( #4592 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-04 16:37:29 -04:00
0727015bf5
HBASE-22749 Distributed MOB compactions ( #4581 )
...
* HBASE-22749 Distributed MOB compactions
- MOB compaction is now handled in-line with per-region compaction on region
servers
- regions with mob data store per-hfile metadata about which mob hfiles are
referenced
- admin requested major compaction will also rewrite MOB files; periodic RS
initiated major compaction will not
- periodically a chore in the master will initiate a major compaction that
will rewrite MOB values to ensure it happens. controlled by
'hbase.mob.compaction.chore.period'. default is weekly
- control how many RS the chore requests major compaction on in parallel
with 'hbase.mob.major.compaction.region.batch.size'. default is as
parallel as possible.
- periodic chore in master will scan backing hfiles from regions to get the
set of referenced mob hfiles and archive those that are no longer
referenced. control period with 'hbase.master.mob.cleaner.period'
- Optionally, RS that are compacting mob files can limit write
amplification by not rewriting values from mob hfiles over a certain size
limit. opt-in by setting 'hbase.mob.compaction.type' to 'optimized'.
control threshold by 'hbase.mob.compactions.max.file.size'.
default is 1GiB
- Should smoothly integrate with existing MOB users via rolling upgrade.
will delay old MOB file cleanup until per-region compaction has managed
to compact each region at least once so that used mob hfile metadata can
be gathered.
* HBASE-22749 Distributed MOB compactions
fix RestrictedApi
Co-authored-by: Vladimir Rodionov <vrodionov@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-06-30 20:44:45 +01:00
a9f9701b72
Revert "HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted"
...
This reverts commit 96693e8b40
2022-06-25 10:57:47 -07:00
e8f04a8a7f
HBASE-27146 Avoid CellUtil.cloneRow in MetaCellComparator ( #4571 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: SiCheng-Zheng <643463623@qq.com>
(cherry picked from commit b1691a5318
2022-06-24 22:37:27 +08:00
da76941b0e
HBASE-27111 Make Netty channel bytebuf allocator configurable. ( #4525 )
...
Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
This allows the user to add a custom implementation, perhaps for debugging.
Also updates ReflectionUtils with a new helper method.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestNettyRpcServer.java
2022-06-23 15:18:32 -07:00
96693e8b40
HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
...
Created OnheapDecodedCell and OffheapDecodedExtendedCell objects with duplicate copy of
ByteBuffer's underlying array instead of original ByteBuffer
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2022-06-13 14:21:03 +02:00
18a1a4bdac
HBASE-27097 SimpleRpcServer is broken ( #4521 )
...
Apply https://issues.apache.org/jira/secure/attachment/13044874/MultiByteBuff.patch
from Lijin Bin on JIRA.
Co-authored-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-11 10:41:38 -07:00
9b8bb580b9
HBASE-26366 Provide meaningful parent spans to ZK interactions
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-10 16:26:37 +02:00
50e2ef24d5
HBASE-27038 CellComparator should extend Serializable ( #4492 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-06 17:45:20 -04:00
40c2743a0c
HBASE-27023 Fix license issues after running spotless:apply ( #4458 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit e555ac4a99
2022-06-02 20:20:43 +08:00
716a56c849
HBASE-27082 Change the return value of RSGroupInfo.getServers from SortedSet to Set to keep compatibility ( #4480 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestUpdateRSGroupConfiguration.java
2022-06-01 09:17:34 -07:00
6dd1a6bef0
HBASE-27081 Fix disallowed compatibility breaks on branch-2.5 and branch-2
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-31 19:56:16 -07:00
71bd62da41
Revert "HBASE-26856:BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted ( #4394 )"
...
This reverts commit de9a17e75c
2022-05-18 13:03:16 +02:00
de9a17e75c
HBASE-26856:BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted ( #4394 )
...
Created OnheapDecodedCell and OffheapDecodedExtendedCell objects with duplicate copy of ByteBuffer's underlying array instead of original ByteBuffer
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit c198f23e5e
2022-05-17 10:58:06 +05:30
43e8c11989
HBASE-27013 Introduce read all bytes when using pread for prefetch ( #4414 )
...
- introduce optional flag `hfile.pread.all.bytes.enabled` for pread that must read full bytes with the next block header
Signed-off-by: Josh Elser <elserj@apache.org>
2022-05-13 13:14:17 -07:00
1a5b1b266c
HBASE-26899 Run spotless:apply
2022-05-01 22:41:49 +08:00
fd2e7205ac
HBASE-22349 slop in StochasticLoadBalancer ( #4371 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java
2022-04-28 12:38:50 -07:00
0f9f6f2cde
HBASE-26959 Brotli compression support ( #4353 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-04-22 16:48:04 -07:00
2240025349
HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException ( #4273 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-04-22 08:56:52 -04:00
e7eb628025
HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless ( #4324 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org
(cherry picked from commit 3ae0d9012c
2022-04-09 23:13:49 +08:00
Duo Zhang
Bryan Beaudreault
Wei-Chiu Chuang
Peter Somogyi
Andor Molnár
mokai
Rushabh Shah
LiangJun He
Bryan Beaudreault
binlijin
Luca Kovács
wenwj0
Andrew Purtell
liangxs
huaxiangsun
Nick Dimiduk
SiCheng-Zheng
chenglei
Bryan Beaudreault
BukrosSzabolcs
Mohammad Arshad
Tak Lon (Stephen) Wu
d-c-manning