Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,426 Commits 95 Branches 866 Tags
Commit Graph

18426 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Duo Zhang
b820188c68 HBASE-27278 Improve TestTlsIPC to reuse existing IPC test code (#4682) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 3309108ca728cb40ad0da5be567ca0e1beabbeff)
 2022-08-12 13:21:11 +08:00
Peter Somogyi
3f5076dac5 HBASE-27281 Add default implementation for Connection$getClusterId (#4683) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-11 17:21:14 +02:00
Duo Zhang
4079647c39 HBASE-27293 Remove jenkins and personality scripts support for 1.x (#4690) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit bffae99f40f5fb144d52450bfedadfd168a429e0)
 2022-08-11 16:59:49 +08:00
Andrew Purtell
2941b8e583 HBASE-27240 Clean up error-prone findings in hbase-http (#4653) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit 5eaeff5fcdd5c4e42376a56cca02e4cbb56cc3f0)
 2022-08-11 14:44:14 +08:00
Masatake Iwasaki
0b9f9871a6 HBASE-27292. Fix build failure against Hadoop 3.3.4 due to added dependency on okhttp. (#4687) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 00612106b5fa78a0dd198cbcaab610bd8b1be277)
 2022-08-10 22:15:23 +08:00
SiCheng-Zheng
25e192b807 HBASE-27248 WALPrettyPrinter add print timestamp (#4659) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6b2e9d14a71d3ccbedabf971fe961802d263a995)
 2022-08-10 21:30:55 +08:00
Bryan Beaudreault
e99faad83e HBASE-27241 Add metrics for evaluating cost and effectiveness of bloom filters (#4669) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-08-09 15:25:47 -04:00
Duo Zhang
beb7c6d490 HBASE-27283 Use readTO instead of hard coded RpcClient.DEFAULT_SOCKET_TIMEOUT_READ when creating ReadTimeoutHandler in NettyRpcConnection (#4685) 
Signed-off-by: Xin Sun <ddupgs@gmail.com
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 5919b30b6d9e905130033461efc37fae60bab947)
 2022-08-09 16:22:57 +08:00
chaijunjie0101
30f6a17056 HBASE-27282 CME in AuthManager causes region server crash (#4684) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit d6641626a97529982720de2b73593e294869c4ca)
 2022-08-09 09:56:26 +08:00
LiangJun He
6e82a26a51 HBASE-27275 graceful_stop.sh unable to restore the balance state (#4680) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b431652e48e7cd003c7f8cad4eb9e3065cb0b40f)
 2022-08-09 09:55:54 +08:00
Abhradeep Kundu
a85de838ab HBASE-27265 : Tool to read StoreFileTrackerFile (#4673) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-08 15:31:14 +01:00
SiCheng-Zheng
c81517638c HBASE-27270 Add bulkloadExecutor.resizeQueues (#4677) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4ebf71982c8a26033c0e7513ef4fc52a303ed71f)
 2022-08-08 18:15:25 +08:00
ryakhovskiy
74726617b8
HBASE-26775 - add synchronized modifier to the toString() method of ProcedureEvent.java (#4681) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-08-08 18:03:40 +08:00
Andor Molnár
2821cc518a HBASE-26666 Add native TLS encryption support to RPC server/client (#4666) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit f8dcf0705538ffeddcaa5eff74c8c2f61f7a844a)

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 2022-08-06 19:43:17 -04:00
Duo Zhang
2e7a983923 HBASE-27273 Should stop autoRead and skip all the bytes when rpc request too big (#4679) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 486d19e99ff4370bc60e0db235508198c84a00e3)
 2022-08-06 20:31:27 +08:00
LiangJun He
c5b22d973b HBASE-27269 The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect (#4678) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 2dc26082be978796046d9124212e8c9d2d8a1f5d)
 2022-08-04 23:14:24 +08:00
Duo Zhang
272805c32e HBASE-27271 BufferCallBeforeInitHandler should ignore the flush request (#4676) 
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
(cherry picked from commit fb529e23526eaa250bdd355db3c2f0dedc55c3e8)

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
 2022-08-04 22:57:13 +08:00
Nick Dimiduk
bcec871959 HBASE-27153 Improvements to read-path tracing 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-08-02 14:09:39 +02:00
Bryan Beaudreault
dbad3d1175 HBASE-27257 Remove unnecessary usage of CachedBlocksByFile from RS UI (#4667) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-07-29 15:28:54 -04:00
Duo Zhang
5e4ddad9a9 HBASE-27185 Addendum fix TestShadeSaslAuthenticationProvider 2022-07-28 17:31:28 +08:00
chenglei
7910f7738f HBASE-27247 TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of ReplicationPeerConfigUtil.parseTableCFsFromConfig (#4658) 
Co-authored-by: comnetwork <comnetwork@163.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-27 17:55:00 -07:00
Andrew Purtell
29f51c05f0
HBASE-27203 Clean up error-prone findings in hbase-client [branch-2] (#4651) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-27 15:58:40 -07:00
Duo Zhang
5f231d3fb3 HBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler (#4624) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 0c4263a18be13f412fcf71ebba9715dcc5ca2a12)

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/security/NettyHBaseSaslRpcClientHandler.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerRpcConnection.java
 2022-07-27 09:10:32 +08:00
Andrew Purtell
6f3538bb4b HBASE-27236 Clean up error-prone warnings in hbase-hbtop (#4649) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-26 18:05:09 -07:00
Andrew Purtell
6e92500444 HBASE-27210 Clean up error-prone findings in hbase-endpoint (#4646) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-endpoint/src/test/java/org/apache/hadoop/hbase/client/TestRpcControllerFactory.java
	hbase-endpoint/src/test/java/org/apache/hadoop/hbase/coprocessor/TestSecureExport.java
 2022-07-26 17:42:32 -07:00
Andrew Purtell
4c08a8a423 HBASE-27087 TestQuotaThrottle times out 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 14:10:49 -07:00
Bryan Beaudreault
827c1ab5dd HBASE-27229 BucketCache statistics should not count evictions by hfile (#4639) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 17:00:46 -04:00
Duo Zhang
84e7cdb2c0 HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061220cc4307ef2ed35291f6fe14bbdb0)
 2022-07-26 23:45:15 +08:00
Bryan Beaudreault
1fd772c3b1 HBASE-27225 Add BucketAllocator bucket size statistic logging (#4637) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-07-25 20:56:05 -04:00
Andrew Purtell
b17bc94b10 HBASE-27235 Clean up error-prone warnings in hbase-hadoop-compat (#4648) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-25 17:30:50 -07:00
Ramón García Fernández
e762b026db HBASE-27179 Issues building with OpenJDK 17 (#4594) 
Fix test case failures in org.apache.hadoop.hbase.http.log.TestLogLevel under Openjdk 17 because of a missing export of java.security.jgss/sun.security.krb5.
Removed option --illegal-access=permit ignored since Openjdk 17.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit e10c15d0306e678ace8748680527e6eecbf424e5)
 2022-07-25 19:23:42 +08:00
Wellington Ramos Chevreuil
80c33e0d46 HBASE-27232 Fix checking for encoded block size when deciding if bloc… (#4640) 
Signed-off-by: Andor Molnár <andor@cloudera.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit d5ed8f59e582272c2e3cd9b6b33488fa0f265cbf)
 2022-07-25 12:08:09 +01:00
Andrew Purtell
eef5edb403 HBASE-27204 BlockingRpcClient will hang for 20 seconds when SASL is enabled after finishing negotiation (#4642) 
Revert "HBASE-24579: Failed SASL authentication does not result in an exception on client side (#1921)"

This reverts commit bd79c4065ccb13a5e217d844376b3e7b9489d2fe.

When Kerberos authentication succeeds, on the server side, after
receiving the final SASL token from the client, we simply wait for
the client to continue by sending the connection header.
After HBASE-24579, on the client side, an additional readStatus()
was added, which mistakenly assumes that after negotiation has
completed a status code will be sent. However when authentication
has succeeded the server will not send one. As a result the client
will hang and only throw an exception when the configured read
timeout is reached, which is 20 seconds by default.

We cannot unilaterally send the expected additional status code
from the server side because older clients will not expect it. The
first call will fail because the client finds unexpected bytes in
the stream ahead of the call response. Fabricating a call response
also does not seem a viable strategy for backwards compatibility.

The HBASE-24579 change needs to be reconsidered given the
difficult backwards compatibility challenges here.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-24 23:06:58 -07:00
Duo Zhang
db98fab114 HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky (#4636) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 31fc97edeede08cfc9c8dd9fe22c3775ede2eebb)

Conflicts:
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestUpdateRSGroupConfiguration.java
 2022-07-21 14:52:39 +08:00
Duo Zhang
468715bff3 Revert "HBASE-27152 Under compaction mark may leak (#4568)" 
This reverts commit 5c5ceb1ef2588b129a5928c22f70c9982eed4a19.
 2022-07-21 09:04:17 +08:00
Xiaolin Ha
5c5ceb1ef2 HBASE-27152 Under compaction mark may leak (#4568) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit da27a67a1e3abfa0117b0cf1a59d01b7a2f8ab05)
 2022-07-20 08:51:58 +08:00
Minwoo Kang
5cf9a9b4b1 HBASE-27219 Change JONI encoding in RegexStringComparator (#4632) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 09:20:55 -07:00
Nihal Jain
eb73492eac HBASE-20499 Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength 
Signed-off-by: tedyu <yuzhihong@gmail.com>
(cherry picked from commit 6d080762ef795adf02dd0ab236c4b3eb73e19a91)
 2022-07-19 13:48:07 +08:00
Duo Zhang
642e933cda HBASE-27144 Addendum fix spotbug warning 2022-07-19 13:41:07 +08:00
Duo Zhang
99f2ab5aa8 HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 10:00:31 +08:00
Duo Zhang
1c56b5fad5 HBASE-27208 Use spotless to purge the missing summary warnings from error prone (#4628) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 7484a9163a438f2cd9c4e0ac5ab88b00b3340737)
 2022-07-19 09:45:42 +08:00
Andrew Purtell
d19805a7bc HBASE-27195 Clean up netty worker/thread pool configuration (#4619) 
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.

Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".

Also, native epoll is not limited to amd64. aarch64 supports it too.

Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 2022-07-18 17:42:37 -07:00
Andrew Purtell
f76c0d9021 HBASE-27202 Clean up error-prone findings in hbase-balancer (#4623) 
On branch-2, the affected code is in hbase-server.

Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/RegionLocationFinder.java
 2022-07-18 16:00:26 -07:00
Andrew Purtell
831a932ec8 HBASE-27200 Clean up error-prone findings in hbase-archetypes (#4621) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-18 15:42:26 -07:00
Duo Zhang
824785eeeb HBASE-27211 Data race in MonitoredTaskImpl could cause split wal failure (#4630) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 9ab0b1504f589c78cd4e01895027d2ce954ba7f7)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/monitoring/TaskMonitor.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 2022-07-18 20:08:38 +08:00
SiCheng-Zheng
325f90bb40 HBASE-27144 Add special rpc handlers for bulkload operations (#4558) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit ff8eb59709225505be41c73678e907759716a700)

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AnnotationReadingPriorityFunction.java
 2022-07-17 21:41:27 +08:00
Bryan Beaudreault
ca15d39026 HBASE-27053 IOException during caching of uncompressed block to the block cache (#4610) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: wenwj0 <wenweijian2@huawei.com>
 2022-07-16 17:33:27 -04:00
Duo Zhang
8d3693a018 HBASE-27192 The retry number for TestSeparateClientZKCluster is too small (#4614) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 648bb6d0eba64f32a90e8b4df20fe50ffe80dd02)
 2022-07-14 08:52:12 +08:00
Andrew Purtell
70a561eb63 HBASE-27194 Add test coverage for SimpleRpcServer (#4616) 
Add test coverage for SimpleRpcServer.

Improve the way we test both SimpleRpcServer and NettyRpcServer. Use
LoadTestKVGenerator to generate random values with varying sizes between
1000 bytes and 1M bytes, and also to verify them when reading the values
back.

Add secure test coverage for both SimpleRpcServer and NettyRpcServer.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-13 11:22:29 -07:00
Andrew Purtell
4e19892a53 HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488) 
* HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements

- Use an async client and work stealing executor for parallelism during loads.
- Remove the verification read retries, these are not that effective during
  replication lag anyway.
- Increase max task attempts because S3 might throttle.
- Implement a side task that exercises Increments by extracting urls from
  content and updating a cf that tracks referrer counts. These are not
  validated at this time. It could be possible to log the increments, sum
  them with a reducer, and then verify the total, but this is left as a
  future exercise.

Signed-off-by: Viraj Jasani <vjasani@apache.org>

* Sum RPC time for writes (loader) and reads (verifier) and mutation bytes submitted. Expose as job counters.

* Fix an issue with completion chaining

* Pause loading if too many operations are in flight
 2022-07-13 09:02:00 -07:00