Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,539 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

1623 Commits

Author SHA1 Message Date
mokai 838638fdab HBASE-27504 Remove duplicated config 'hbase.normalizer.merge.min_region_age.days' in hbase-default.xml (#4894) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit c691091f78)
 2022-11-23 20:43:42 +08:00
Rushabh Shah ecf3debd42
HBASE-26913 Replication Observability Framework (#4862) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-11-08 13:05:42 -08:00
Duo Zhang 738ed028ad HBASE-27434 Use ${revision} as placeholder for maven version to make it easier to control the version from command line (#4836) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 2fc879e863)
 2022-10-24 12:05:01 +08:00
Duo Zhang 5cab4be075 HBASE-27401 Clean up current broken 'n's in our javadoc (#4812) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 63cdd026f0)

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionLocation.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestReplication.java
	hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteTable.java
	hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/HBaseServerBase.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/RegionServerFlushTableProcedureManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSource.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtil.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestBulkLoadHFilesSplitRecovery.java
	hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java
 2022-10-06 22:13:14 +08:00
LiangJun He f69b47275e
HBASE-27304 Support using IP to expose master/rs servers for some special scenarios (#4798) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-09-25 22:50:06 +08:00
Bryan Beaudreault 2a92819f0b HBASE-27280 Add mutual authentication support to TLS (#4796) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Andor Molnár <andor@cloudera.com>
 2022-09-23 09:11:30 -04:00
Duo Zhang 9fb3aee22b
HBASE-27373 Fix new spotbugs warnings after upgrading spotbugs to 4.7.2 (#4787) (#4789) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-09-18 18:31:05 +08:00
binlijin bfd8501e23
HBASE-27314 Make index block be customized and configured (#4763) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-09-08 09:27:53 +08:00
Andor Molnár c3a30232a8 HBASE-27346 Autodetect key/truststore file type from file extension (#4757) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2022-09-06 16:50:54 -04:00
Andor Molnár 9838c070aa
HBASE-27342 Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores (#4751) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
 2022-09-01 10:06:15 +02:00
Duo Zhang 935b0d2277 HBASE-27321 The ReplicationLogCleaner is not thread safe but can be called from different threads at the same time (#4730) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 37651ee1b0)
 2022-08-28 00:30:34 +08:00
Luca Kovács c04edf7835 HBASE-20904 Prometheus /metrics http endpoint for monitoring (#4691) 
Co-authored-by: Luca Kovacs <kovacs.luca.agota@gmail.com>
Co-authored-by: Madhusoodan P <akshayapataki123@gmail.com>
Co-authored-by: Luca Kovacs <lkovacs@cloudera.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f9ea7ee0d6)

Conflicts:
	src/main/asciidoc/_chapters/ops_mgt.adoc
 2022-08-24 11:31:16 +08:00
wenwj0 f509e43eaa HBASE-27089 Add “commons.crypto.stream.buffer.size” configuration (#4491) 
Co-authored-by: wenwj0 <wenweijian2@huawei.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 7ff927dbd8)
 2022-08-21 22:32:39 +08:00
Andrew Purtell f705e3e7d8 HBASE-27206 Clean up error-prone findings in hbase-common (#4645) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fbe3b90e0c)

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/TestKeyStoreKeyProvider.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/types/TestOrderedFloat32.java
 2022-08-21 21:36:01 +08:00
liangxs aaad3a7077 HBASE-26982 Add index and bloom filter statistics of LruBlockCache on… (#4376) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: stack <stack@apache.org>
 2022-08-15 11:12:05 -07:00
huaxiangsun b25ded026b
HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose (#4695) (#4701) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-08-15 09:16:29 -07:00
Andrew Purtell f36b34984f
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#4617) 
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446)

* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.

Signed-off-by: Esteban Gutierrez <esteban@apache.org>

* spotless:apply to fix HBaseTestingUtility

* Fix error-prone errors

Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>
 2022-08-12 14:29:26 -07:00
Duo Zhang b820188c68 HBASE-27278 Improve TestTlsIPC to reuse existing IPC test code (#4682) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 3309108ca7)
 2022-08-12 13:21:11 +08:00
Andor Molnár 2821cc518a HBASE-26666 Add native TLS encryption support to RPC server/client (#4666) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit f8dcf07055)

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 2022-08-06 19:43:17 -04:00
Nick Dimiduk bcec871959 HBASE-27153 Improvements to read-path tracing 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-08-02 14:09:39 +02:00
Duo Zhang 84e7cdb2c0 HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061)
 2022-07-26 23:45:15 +08:00
Duo Zhang 99f2ab5aa8 HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 10:00:31 +08:00
Andrew Purtell d19805a7bc HBASE-27195 Clean up netty worker/thread pool configuration (#4619) 
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.

Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".

Also, native epoll is not limited to amd64. aarch64 supports it too.

Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 2022-07-18 17:42:37 -07:00
SiCheng-Zheng 325f90bb40 HBASE-27144 Add special rpc handlers for bulkload operations (#4558) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit ff8eb59709)

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AnnotationReadingPriorityFunction.java
 2022-07-17 21:41:27 +08:00
chenglei e7551c1359
HBASE-26950 Use AsyncConnection in ReplicationSink (#4595) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2022-07-09 10:25:17 +08:00
Bryan Beaudreault 57a3781160 HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse (#4592) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-04 16:37:29 -04:00
BukrosSzabolcs 0727015bf5
HBASE-22749 Distributed MOB compactions (#4581) 
* HBASE-22749 Distributed MOB compactions

- MOB compaction is now handled in-line with per-region compaction on region
  servers
- regions with mob data store per-hfile metadata about which mob hfiles are
  referenced
- admin requested major compaction will also rewrite MOB files; periodic RS
  initiated major compaction will not
- periodically a chore in the master will initiate a major compaction that
  will rewrite MOB values to ensure it happens. controlled by
  'hbase.mob.compaction.chore.period'. default is weekly
- control how many RS the chore requests major compaction on in parallel
  with 'hbase.mob.major.compaction.region.batch.size'. default is as
  parallel as possible.
- periodic chore in master will scan backing hfiles from regions to get the
  set of referenced mob hfiles and archive those that are no longer
  referenced. control period with 'hbase.master.mob.cleaner.period'
- Optionally, RS that are compacting mob files can limit write
  amplification by not rewriting values from mob hfiles over a certain size
  limit. opt-in by setting 'hbase.mob.compaction.type' to 'optimized'.
  control threshold by 'hbase.mob.compactions.max.file.size'.
  default is 1GiB
- Should smoothly integrate with existing MOB users via rolling upgrade.
  will delay old MOB file cleanup until per-region compaction has managed
  to compact each region at least once so that used mob hfile metadata can
  be gathered.

* HBASE-22749 Distributed MOB compactions

fix RestrictedApi

Co-authored-by: Vladimir Rodionov <vrodionov@apache.org>

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-06-30 20:44:45 +01:00
Andrew Purtell a9f9701b72 Revert "HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted" 
This reverts commit 96693e8b40.

See also HBASE-27151 TestMultiRespectsLimits.testBlockMultiLimits repeatable failure
 2022-06-25 10:57:47 -07:00
Duo Zhang e8f04a8a7f HBASE-27146 Avoid CellUtil.cloneRow in MetaCellComparator (#4571) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: SiCheng-Zheng <643463623@qq.com>
(cherry picked from commit b1691a5318)
 2022-06-24 22:37:27 +08:00
Andrew Purtell da76941b0e HBASE-27111 Make Netty channel bytebuf allocator configurable. (#4525) 
Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
   preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
  name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
  This allows the user to add a custom implementation, perhaps for debugging.

Also updates ReflectionUtils with a new helper method.

Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestNettyRpcServer.java
 2022-06-23 15:18:32 -07:00
Mohammad Arshad 96693e8b40 HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted 
Created OnheapDecodedCell and OffheapDecodedExtendedCell objects with duplicate copy of
ByteBuffer's underlying array instead of original ByteBuffer

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2022-06-13 14:21:03 +02:00
Andrew Purtell 18a1a4bdac HBASE-27097 SimpleRpcServer is broken (#4521) 
Apply https://issues.apache.org/jira/secure/attachment/13044874/MultiByteBuff.patch
from Lijin Bin on JIRA.

Co-authored-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-06-11 10:41:38 -07:00
Nick Dimiduk 9b8bb580b9
HBASE-26366 Provide meaningful parent spans to ZK interactions 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-10 16:26:37 +02:00
Bryan Beaudreault 50e2ef24d5 HBASE-27038 CellComparator should extend Serializable (#4492) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-06 17:45:20 -04:00
Duo Zhang 40c2743a0c HBASE-27023 Fix license issues after running spotless:apply (#4458) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit e555ac4a99)
 2022-06-02 20:20:43 +08:00
Duo Zhang 716a56c849 HBASE-27082 Change the return value of RSGroupInfo.getServers from SortedSet to Set to keep compatibility (#4480) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestUpdateRSGroupConfiguration.java
 2022-06-01 09:17:34 -07:00
Andrew Purtell 6dd1a6bef0 HBASE-27081 Fix disallowed compatibility breaks on branch-2.5 and branch-2 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-31 19:56:16 -07:00
Nick Dimiduk 71bd62da41 Revert "HBASE-26856:BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted (#4394)" 
This reverts commit de9a17e75c.
 2022-05-18 13:03:16 +02:00
Mohammad Arshad de9a17e75c HBASE-26856:BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted (#4394) 
Created OnheapDecodedCell and OffheapDecodedExtendedCell objects with duplicate copy of ByteBuffer's underlying array instead of original ByteBuffer

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit c198f23e5e)
 2022-05-17 10:58:06 +05:30
Tak Lon (Stephen) Wu 43e8c11989 HBASE-27013 Introduce read all bytes when using pread for prefetch (#4414) 
- introduce optional flag `hfile.pread.all.bytes.enabled` for pread that must read full bytes with the next block header

Signed-off-by: Josh Elser <elserj@apache.org>
 2022-05-13 13:14:17 -07:00
Duo Zhang 1a5b1b266c HBASE-26899 Run spotless:apply 2022-05-01 22:41:49 +08:00
d-c-manning fd2e7205ac HBASE-22349 slop in StochasticLoadBalancer (#4371) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java
 2022-04-28 12:38:50 -07:00
Andrew Purtell 0f9f6f2cde HBASE-26959 Brotli compression support (#4353) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-04-22 16:48:04 -07:00
Bryan Beaudreault 2240025349
HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException (#4273) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-04-22 08:56:52 -04:00
Duo Zhang e7eb628025 HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless (#4324) 
Signed-off-by: Yulin Niu <niuyulin@apache.org
(cherry picked from commit 3ae0d9012c)
 2022-04-09 23:13:49 +08:00
Duo Zhang 197b8d48ec HBASE-26928 Fix several indentation problems (#4323) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit e68c61dd54)
 2022-04-06 15:14:03 +08:00
chenglei 5affa20359
HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null (#4264) 2022-03-23 18:31:47 +08:00
chenglei c55c289e99
Revert "HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null (#4262)" (#4263) 
This reverts commit d889746bc6.
 2022-03-23 17:52:11 +08:00
chenglei d889746bc6
HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null (#4262) 2022-03-23 17:48:20 +08:00
Duo Zhang 9e1de13d03 HBASE-25465 Use javac --release option for supporting cross version compilation (#4164) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-12 18:30:29 +08:00