Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,466 Commits 95 Branches 866 Tags
Commit Graph

8890 Commits

Author SHA1 Message Date
chenglei
88d1bd0942 HBASE-27362 CompactSplit.requestCompactionInternal may bypass compactionsEnabled check (#4768) 
Co-authored-by: comnetwork <comnetwork@163.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit a26cbf105743de28c62a6442eba2ef04d17d46af)
 2022-09-08 16:43:52 +08:00
binlijin
bfd8501e23
HBASE-27314 Make index block be customized and configured (#4763) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-09-08 09:27:53 +08:00
Bryan Beaudreault
b103685a1b HBASE-27224 HFile tool statistic sampling produces misleading results (#4638) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by:  Clay Baenziger <cwb@clayb.net>
 2022-09-06 15:23:18 -04:00
Xiaolin Ha
18d03058aa HBASE-27332 Remove RejectedExecutionHandler for long/short compaction thread pools (#4731) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-09-05 21:14:08 +08:00
Michael Stack
0f29423cc5 HBASE-27340 Artifacts with resolved profiles (#4740) 
Make it so our published poms carry the minimum needed to run
    an hbase; the published pom has no profiles -- the profiles
    specified at build time are resolved, their dependencies inlined,
    and then they are stripped -- and no build-time, or plugins
    dependencies or properties, etc. Resultant poms have explicit
    hadoop lib versions baked in -- no more being able to choose
    hbase with hadoop2 or haddop3 at downstream build time by setting
    a '-Dhadoop.profile=X.0'.

    Pattern is to add profiles when none in sub-modules when
    the flatten plugin complains it can't resolve an hadoop
    dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
    Adding the hadoop-2.0 and hadoop-3.0 profiles in the sub-module
    make it so the flatten plugin can figure 'hadoop.version'
    definitively.

    Another spin on the above happens when profiles already exist
    in submodule but the flatten plugin is complaining it can't
    figure figure version on an hadoop dependency NOT under
    profiles. Below, we move the delinquent hadoop dependency under
    existing profiles (minikdc was the usual dependency outside
    profiles in sub-modules that flatten complained about).

    Sometimes, moving an hadoop dependency under a profile, there
    would be excludes on the local dependency. If the parent pom
    excludes section was missing the local excludes, we added them
    up to the parent module so all excluding is done up there in
    the parent profile dependencyManagement section.

    * hbase-asyncfs/pom.xml
    * hbase-endpoint/pom.xml
    * hbase-examples/pom.xml
    * hbase-http/pom.xml
    * hbase-rest/pom.xml
    * hbase-server/pom.xml
     Move the minikdc under profiles so it picks up appropriate hadoop version
     when the flatten plugin runs.

    * hbase-hadoop2-compat/pom.xml
     Add hadoop2 and hadoop3 profiles and move hadoop-common, etc.
     under them so we pick up appropriate hadoop version when flatten
     plugin runs.

    * hbase-mapreduce/pom.xml
     Move hadoop dependencies under profiles so right version is
     available when the flatten plugin runs.

    * hbase-shaded/hbase-shaded-testing-util/pom.xml
     Add profiles for hadoop-2.0 and hadoop-3.0 and move the
     hadoop dependencies under them.

    pom.xml
     Add the flatten plugin with the flatten profiles enabled.
     Add a few excludes on hadoop profiles picked up from sub-modules.
     E.g. exclude bouncycastle bcprov-jdk15 when we include minikdc.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-09-02 16:44:27 -07:00
Andor Molnár
9838c070aa
HBASE-27342 Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores (#4751) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
 2022-09-01 10:06:15 +02:00
Istvan Toth
a7dab37b7f
HBASE-22939 SpaceQuotas - Bulkload from different hdfs failed when space quotas are turned on. (#4748) 
Signed-off-by: Sakthi <sakthi@apache.org>
 2022-08-31 15:47:52 +02:00
Xiaolin Ha
bd032460b8
HBASE-27152 Under compaction mark may leak (#4725) (#4744) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-30 01:19:09 +08:00
LiangJun He
2e53a517a2 HBASE-27336 The region visualizer shows 'undefined' region server (#4736) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fa02350a27d527a6e83a5b50ca80577847dfbb5f)
 2022-08-28 22:52:49 +08:00
Duo Zhang
935b0d2277 HBASE-27321 The ReplicationLogCleaner is not thread safe but can be called from different threads at the same time (#4730) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 37651ee1b0c019b2637142aabb8e17eb6d1d3c84)
 2022-08-28 00:30:34 +08:00
Ujjawal
c71e984a1f HBASE-25922 - Disabled sanity checks ignored on snapshot restore (#4533) 
Co-authored-by: Ujjawal Kumar <u.kumar@ukumar-ltmit1s.internal.salesforce.com>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 2022-08-26 08:57:12 -04:00
Ruanhui
d92d533924 HBASE-27325 the bulkload max call queue size can be update to a wrong value (#4728) 
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 92cf962fee37f83dc7fafbb5571a699bf23ab275)
 2022-08-25 11:00:28 +08:00
Ruanhui
f66d67bbdf HBASE-27305 add an option to skip file splitting when bulkload hfiles (#4709) 
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 00a719e76f16b7380f6695fc986f003e0e5f47fe)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java
 2022-08-22 22:45:30 +08:00
LiangJun He
1bd0b581cb
HBASE-27104 Add a tool command list_unknownservers (#4714) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-22 21:15:47 +08:00
tianhang
fc93fbac9a HBASE-27129 Add a config that allows us to configure region-level storage policies (#4547) 
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f238a92b6dfc63dd6a14ea49a7073d6f5ee01a73)
 2022-08-22 10:07:27 +08:00
Andrew Purtell
f705e3e7d8 HBASE-27206 Clean up error-prone findings in hbase-common (#4645) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fbe3b90e0c4eef1dc13fb2a5ed9381106ca671dd)

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/TestKeyStoreKeyProvider.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/types/TestOrderedFloat32.java
 2022-08-21 21:36:01 +08:00
Fantasy-Jay
8a10dbf6c4 HBASE-27268 In trace log mode, the client does not print callId/startTime and the server does not print receiveTime (#4710) 
Co-authored-by: jay.zhu <jay.zhu@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 32c135d8a552970e592ed39735186b4364f85ed1)
 2022-08-20 21:09:43 +08:00
BukrosSzabolcs
d073404706 HBASE-26969:Eliminate MOB renames when SFT is enabled 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>

closes #4712
 2022-08-19 11:33:48 +02:00
Peter Somogyi
1176b484d1 HBASE-25970 MOB data loss - incorrect concatenation of MOB_FILE_REFS (#3355) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2022-08-19 11:20:53 +02:00
Sean Busbey
1986a6ec31 HBASE-24163 MOB compactor implementations should use format specifiers when calling String.format 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-08-17 13:46:18 +02:00
Duo Zhang
c2dff398d3 HBASE-27279 Make SslHandler work with SaslWrapHandler/SaslUnwrapHandler (#4705) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: Andor Molnár <andor@cloudera.com>
(cherry picked from commit 2b9d36869ffe765256d4a7b6a74b95a8200678df)

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/security/TestNettyTlsIPCRejectPlainText.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/security/TestSecureIPC.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/security/TestTlsWithKerberos.java
 2022-08-16 21:37:47 +08:00
Wellington Ramos Chevreuil
3acf920e1f HBASE-27264 Add options to consider compressed size when delimiting blocks during hfile writes (#4675) 
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
 2022-08-15 22:36:44 +01:00
liangxs
aaad3a7077 HBASE-26982 Add index and bloom filter statistics of LruBlockCache on… (#4376) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: stack <stack@apache.org>
 2022-08-15 11:12:05 -07:00
huaxiangsun
b25ded026b
HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose (#4695) (#4701) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-08-15 09:16:29 -07:00
Andrew Purtell
f36b34984f
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#4617) 
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446)

* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.

Signed-off-by: Esteban Gutierrez <esteban@apache.org>

* spotless:apply to fix HBaseTestingUtility

* Fix error-prone errors

Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>
 2022-08-12 14:29:26 -07:00
Duo Zhang
b820188c68 HBASE-27278 Improve TestTlsIPC to reuse existing IPC test code (#4682) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 3309108ca728cb40ad0da5be567ca0e1beabbeff)
 2022-08-12 13:21:11 +08:00
SiCheng-Zheng
25e192b807 HBASE-27248 WALPrettyPrinter add print timestamp (#4659) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6b2e9d14a71d3ccbedabf971fe961802d263a995)
 2022-08-10 21:30:55 +08:00
Bryan Beaudreault
e99faad83e HBASE-27241 Add metrics for evaluating cost and effectiveness of bloom filters (#4669) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-08-09 15:25:47 -04:00
Duo Zhang
beb7c6d490 HBASE-27283 Use readTO instead of hard coded RpcClient.DEFAULT_SOCKET_TIMEOUT_READ when creating ReadTimeoutHandler in NettyRpcConnection (#4685) 
Signed-off-by: Xin Sun <ddupgs@gmail.com
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 5919b30b6d9e905130033461efc37fae60bab947)
 2022-08-09 16:22:57 +08:00
chaijunjie0101
30f6a17056 HBASE-27282 CME in AuthManager causes region server crash (#4684) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit d6641626a97529982720de2b73593e294869c4ca)
 2022-08-09 09:56:26 +08:00
Abhradeep Kundu
a85de838ab HBASE-27265 : Tool to read StoreFileTrackerFile (#4673) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-08 15:31:14 +01:00
SiCheng-Zheng
c81517638c HBASE-27270 Add bulkloadExecutor.resizeQueues (#4677) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4ebf71982c8a26033c0e7513ef4fc52a303ed71f)
 2022-08-08 18:15:25 +08:00
Andor Molnár
2821cc518a HBASE-26666 Add native TLS encryption support to RPC server/client (#4666) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit f8dcf0705538ffeddcaa5eff74c8c2f61f7a844a)

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 2022-08-06 19:43:17 -04:00
Duo Zhang
2e7a983923 HBASE-27273 Should stop autoRead and skip all the bytes when rpc request too big (#4679) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 486d19e99ff4370bc60e0db235508198c84a00e3)
 2022-08-06 20:31:27 +08:00
LiangJun He
c5b22d973b HBASE-27269 The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect (#4678) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 2dc26082be978796046d9124212e8c9d2d8a1f5d)
 2022-08-04 23:14:24 +08:00
Nick Dimiduk
bcec871959 HBASE-27153 Improvements to read-path tracing 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-08-02 14:09:39 +02:00
Bryan Beaudreault
dbad3d1175 HBASE-27257 Remove unnecessary usage of CachedBlocksByFile from RS UI (#4667) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-07-29 15:28:54 -04:00
Duo Zhang
5f231d3fb3 HBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler (#4624) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 0c4263a18be13f412fcf71ebba9715dcc5ca2a12)

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/security/NettyHBaseSaslRpcClientHandler.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerRpcConnection.java
 2022-07-27 09:10:32 +08:00
Andrew Purtell
4c08a8a423 HBASE-27087 TestQuotaThrottle times out 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 14:10:49 -07:00
Bryan Beaudreault
827c1ab5dd HBASE-27229 BucketCache statistics should not count evictions by hfile (#4639) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 17:00:46 -04:00
Duo Zhang
84e7cdb2c0 HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061220cc4307ef2ed35291f6fe14bbdb0)
 2022-07-26 23:45:15 +08:00
Bryan Beaudreault
1fd772c3b1 HBASE-27225 Add BucketAllocator bucket size statistic logging (#4637) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-07-25 20:56:05 -04:00
Wellington Ramos Chevreuil
80c33e0d46 HBASE-27232 Fix checking for encoded block size when deciding if bloc… (#4640) 
Signed-off-by: Andor Molnár <andor@cloudera.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit d5ed8f59e582272c2e3cd9b6b33488fa0f265cbf)
 2022-07-25 12:08:09 +01:00
Duo Zhang
db98fab114 HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky (#4636) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 31fc97edeede08cfc9c8dd9fe22c3775ede2eebb)

Conflicts:
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestUpdateRSGroupConfiguration.java
 2022-07-21 14:52:39 +08:00
Duo Zhang
468715bff3 Revert "HBASE-27152 Under compaction mark may leak (#4568)" 
This reverts commit 5c5ceb1ef2588b129a5928c22f70c9982eed4a19.
 2022-07-21 09:04:17 +08:00
Xiaolin Ha
5c5ceb1ef2 HBASE-27152 Under compaction mark may leak (#4568) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit da27a67a1e3abfa0117b0cf1a59d01b7a2f8ab05)
 2022-07-20 08:51:58 +08:00
Nihal Jain
eb73492eac HBASE-20499 Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength 
Signed-off-by: tedyu <yuzhihong@gmail.com>
(cherry picked from commit 6d080762ef795adf02dd0ab236c4b3eb73e19a91)
 2022-07-19 13:48:07 +08:00
Duo Zhang
642e933cda HBASE-27144 Addendum fix spotbug warning 2022-07-19 13:41:07 +08:00
Duo Zhang
99f2ab5aa8 HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 10:00:31 +08:00
Andrew Purtell
d19805a7bc HBASE-27195 Clean up netty worker/thread pool configuration (#4619) 
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.

Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".

Also, native epoll is not limited to amd64. aarch64 supports it too.

Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 2022-07-18 17:42:37 -07:00