2478 Commits

Author SHA1 Message Date
LiangJun He
78f587f559 HBASE-27104 Add a tool command list_unknownservers (#4714)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 1bd0b581cb68341d6087f9fda0e4f3efe289a3a0)
2022-08-22 21:26:19 +08:00
Fantasy-Jay
46ffdabd13 HBASE-27268 In trace log mode, the client does not print callId/startTime and the server does not print receiveTime (#4710)
Co-authored-by: jay.zhu <jay.zhu@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 32c135d8a552970e592ed39735186b4364f85ed1)
2022-08-20 21:09:53 +08:00
SiCheng-Zheng
0ceb3577fc HBASE-27301 Add Delete addFamilyVersion timestamp verify (#4700)
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b531e71455423a9f5a5956da90026085dde96117)
2022-08-15 15:23:34 +08:00
Peter Somogyi
ad6770652f HBASE-27281 Add default implementation for Connection$getClusterId (#4683)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-11 17:21:33 +02:00
Duo Zhang
187c084cf2 HBASE-27283 Use readTO instead of hard coded RpcClient.DEFAULT_SOCKET_TIMEOUT_READ when creating ReadTimeoutHandler in NettyRpcConnection (#4685)
Signed-off-by: Xin Sun <ddupgs@gmail.com
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 5919b30b6d9e905130033461efc37fae60bab947)
2022-08-09 16:23:08 +08:00
Duo Zhang
1720ced4c5 HBASE-27271 BufferCallBeforeInitHandler should ignore the flush request (#4676)
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
(cherry picked from commit fb529e23526eaa250bdd355db3c2f0dedc55c3e8)

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
2022-08-04 23:03:52 +08:00
Nick Dimiduk
472e81098e HBASE-27153 Improvements to read-path tracing
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-08-02 14:10:05 +02:00
chenglei
95a2e59618 HBASE-27247 TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of ReplicationPeerConfigUtil.parseTableCFsFromConfig (#4658)
Co-authored-by: comnetwork <comnetwork@163.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-07-27 17:57:09 -07:00
Andrew Purtell
6a169b5079 HBASE-27203 Clean up error-prone findings in hbase-client [branch-2] (#4651)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-27 15:59:06 -07:00
Duo Zhang
88f957b323 HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061220cc4307ef2ed35291f6fe14bbdb0)
2022-07-26 23:45:29 +08:00
Andrew Purtell
9cec74c503 HBASE-27204 BlockingRpcClient will hang for 20 seconds when SASL is enabled after finishing negotiation (#4642)
Revert "HBASE-24579: Failed SASL authentication does not result in an exception on client side (#1921)"

This reverts commit bd79c4065ccb13a5e217d844376b3e7b9489d2fe.

When Kerberos authentication succeeds, on the server side, after
receiving the final SASL token from the client, we simply wait for
the client to continue by sending the connection header.
After HBASE-24579, on the client side, an additional readStatus()
was added, which mistakenly assumes that after negotiation has
completed a status code will be sent. However when authentication
has succeeded the server will not send one. As a result the client
will hang and only throw an exception when the configured read
timeout is reached, which is 20 seconds by default.

We cannot unilaterally send the expected additional status code
from the server side because older clients will not expect it. The
first call will fail because the client finds unexpected bytes in
the stream ahead of the call response. Fabricating a call response
also does not seem a viable strategy for backwards compatibility.

The HBASE-24579 change needs to be reconsidered given the
difficult backwards compatibility challenges here.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-24 23:07:01 -07:00
Minwoo Kang
ee56bcafb4 HBASE-27219 Change JONI encoding in RegexStringComparator (#4632)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 09:20:59 -07:00
Duo Zhang
9c3528d730 HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 10:03:26 +08:00
chenglei
8d0776c7b8
HBASE-26950 Use AsyncConnection in ReplicationSink (#4607)
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2022-07-09 11:29:05 +08:00
Bryan Beaudreault
94fc45ef76 HBASE-27078 Allow configuring a separate timeout for meta scans (#4585)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-07 16:48:28 -04:00
Duo Zhang
3ca8484c56 HBASE-26708 Netty leak detected and OutOfDirectMemoryError due to direct memory buffering with SASL implementation (#4596)
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 816e919e952de72c94543180870a438ff3c53caa)
2022-07-07 16:04:33 +08:00
Duo Zhang
baddedf783 HBASE-23330: Fix delegation token fetch with MasterRegistry (#1084) (#4598)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit d8b3f55518fcf73df54f67ab1cb2f3920088d70d)

Co-authored-by: Bharath Vissapragada <bharathv@apache.org>
2022-07-06 11:54:00 +08:00
Bryan Beaudreault
2b36963d46 HBASE-26790 getAllRegionLocations can cache locations with null hostname (#4575)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-24 07:41:55 -04:00
LiangJun He
53b57d686b HBASE-27028 Add a shell command for flushing master local region (#4539)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 666aa064e7cfe8efa2c020a6831206b71df0d8a9)
2022-06-21 13:35:38 +08:00
Nick Dimiduk
ac7622c398
HBASE-26366 Provide meaningful parent spans to ZK interactions
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-13 10:33:49 +02:00
wangzhi
5cc614f23f HBASE-27093 AsyncNonMetaRegionLocator:put Complete CompletableFuture outside lock block (#4496)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 176c43c5ad1aab01eb2d2b05c0cb90132e8d19b1)
2022-06-07 12:18:46 +08:00
huaxiangsun
5a13ad8137
HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4442) (#4484)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-03 15:27:00 -07:00
Andrew Purtell
b7158a87ea Preparing development version 2.5.1-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-31 20:06:32 -07:00
Andrew Purtell
2da2dd917f Preparing hbase release 2.5.0RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-31 20:06:29 -07:00
Andrew Purtell
4d6636dfea HBASE-27081 Fix disallowed compatibility breaks on branch-2.5 and branch-2
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-31 19:53:58 -07:00
Nick Dimiduk
c47a2c3841 HBASE-26986 Trace a one-shot execution of a Master procedure
This one adds a tracing context to the Master's ProcedureV2 execution. This ties all the spans
that result from a procedure execution back to a procedure by name. Maybe in a follow-on issue, we
can serialize span context along with procedures and display procedure start/stop/retry events and
parent-child hierarchy in tracing visualization.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-05-18 13:08:42 +02:00
Bryan Beaudreault
341ce509fe HBASE-26990 Add default implementation for BufferedMutator interface setters (#4387)
Signed-off-by: Josh Elser <elserj@apache.org>
2022-05-03 15:12:01 -04:00
Nick Dimiduk
ede80fb108 HBASE-26648 Improve fidelity of RegionLocator spans
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-05-03 13:47:35 +02:00
Duo Zhang
1aea663c6d HBASE-26899 Run spotless:apply 2022-05-01 22:52:40 +08:00
Bryan Beaudreault
4fb47b4800 HBASE-26807 Addendum: adhere to maxAttempts in RawAsyncTableImpl.batch (#4370)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-04-25 13:25:59 -04:00
Ruanhui
e2ac5a9338 HBASE-26942 cache region locations when getAllRegionLocations (#4364)
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-04-22 15:31:53 -04:00
Bryan Beaudreault
6fae190413 HBASE-26891 Make MetricsConnection scope configurable (#4365)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-04-22 11:57:53 -04:00
Bryan Beaudreault
dc86de467b HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException (#4273)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-04-22 08:58:35 -04:00
huaxiangsun
43af814d1a
HBASE-26618 Involving primary meta region in meta scan with CatalogRe… (#4321) (#4327)
Signed-off-by: Michael Stack <stack@apache.org>
2022-04-11 10:28:55 -07:00
Duo Zhang
913dd9c305 HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless (#4324)
Signed-off-by: Yulin Niu <niuyulin@apache.org
(cherry picked from commit 3ae0d9012c6a0132147f2cd1a96337213bbf2b8d)
2022-04-09 23:21:21 +08:00
Andrew Purtell
abb1bf2617 Revert "HBASE-26306 Backport "HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node" to branch-2 (#3727)"
This reverts commit 10584d70d254239f2af05ec952c4cc53762acf05.

See HBASE-26937 for discussion.

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnection.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
2022-04-08 09:14:01 -07:00
Nick Dimiduk
235308d8bf HBASE-26545 Implement tracing of scan
* on `AsyncTable`, both `scan` and `scanAll` methods should result in `SCAN` table operations.
* the span of the `SCAN` table operation should have children representing all the RPC calls
  involved in servicing the scan.
* when a user provides custom implementation of `AdvancedScanResultConsumer`, any spans emitted
  from the callback methods should also be tied to the span that represents the `SCAN` table
  operation. This is easily done because these callbacks are executed on the RPC thread.
* when a user provides a custom implementation of `ScanResultConsumer`, any spans emitted from the
  callback methods should be also be tied to the span that represents the `SCAN` table
  operation. This accomplished by carefully passing the span instance around after it is created.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-04-01 12:47:33 +02:00
chenglei
69ea6f579f
HBASE-26811 Secondary replica may be disabled for read incorrectly forever (#4309) 2022-04-01 11:28:29 +08:00
chenglei
2bf81c6647
HBASE-26812 ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls (#4307) 2022-04-01 10:42:40 +08:00
Andrew Purtell
6902cb2568 HBASE-26826 Backport StoreFileTracker (HBASE-26067, HBASE-26584, and others) to branch-2.5
Previous cherry picks:

commit 6aaef89 HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic
commit 43b40e9 HBASE-25988 Store the store file list by a file #3578)
commit 6e05376 HBASE-26079 Use StoreFileTracker when splitting and merging #3617)
commit 090b2fe HBASE-26224 HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin… #3656)
commit 0ee1689 HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table #3666)
commit 2052e80 HBASE-26248 Should find a suitable way to let users specify the store… #3665)
commit 5ff0f98 HBASE-26264 Add more checks to prevent misconfiguration on store file… #3681)
commit fc4f6d1 HBASE-26280 HBASE-26280 Use store file tracker when snapshoting #3685)
commit 06db852 HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… #3721)
commit e4e7cf8 HBASE-26386 Refactor StoreFileTracker implementations to expose the s… #3774)
commit 08d1171 HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl #3749)
commit 8bec26e HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur… #3700)
commit a288365 HBASE-26271: Cleanup the broken store files under data directory #3786)
commit d00b5fa HBASE-26454 CreateTableProcedure still relies on temp dir and renames… #3845)
commit 771e552 HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
commit f16b7b1 HBASE-26265 Update ref guide to mention the new store file tracker im… #3942)
commit 755b3b4 HBASE-26585 Add SFT configuration to META table descriptor when creating META #3998)
commit 39c42c7 HBASE-26639 The implementation of TestMergesSplitsAddToTracker is pro… #4010)
commit 6e1f5b7 HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading #4006)
commit f1dd865 HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor… #4034)
commit 8fbc9a2 HBASE-26674 Should modify filesCompacting under storeWriteLock #4040)
commit 5aa0fd2 HBASE-26675 Data race on Compactor.writer #4035)
commit 3021c58 HBASE-26700 The way we bypass broken track file is not enough in Stor… #4055)
commit a8b68c9 HBASE-26690 Modify FSTableDescriptors to not rely on renaming when wr… #4054)
commit dffeb8e HBASE-26587 Introduce a new Admin API to change SFT implementation (#… #4080)
commit b265fe5 HBASE-26673 Implement a shell command for change SFT implementation #4113)
commit 4cdb380 HBASE-26640 Reimplement master local region initialization to better … #4111)
commit 77bb153 HBASE-26707: Reduce number of renames during bulkload (#4066) #4122)
commit a4b192e HBASE-26611 Changing SFT implementation on disabled table is dangerous #4082)
commit d3629bb HBASE-26837 Set SFT config when creating TableDescriptor in TestClone… #4226)
commit 541d748 HBASE-26881 Backport HBASE-25368 to branch-2 (#4267)

Fixups for precommit error prone, checkstyle, and javadoc warnings after applying cherry picks.

Signed-off-by: Josh Elser <elserj@apache.org>
Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil
2bbddfee1e HBASE-26881 Backport HBASE-25368 to branch-2 (#4267)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang
6c43e62163 HBASE-26587 Introduce a new Admin API to change SFT implementation (#4030) (#4080)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Reviewed-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
BukrosSzabolcs
16ad5f777d HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
Closes #3851

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
GeorryHuang
baac21b047 HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables (#3700)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil
b80efeae09 HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table (#3666)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-26 09:30:58 -07:00
Bryan Beaudreault
cfd0a1786a HBASE-26876 Use toStringBinary for rowkey in RegionServerCallable error string (#4269)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-26 06:49:42 +08:00
Bryan Beaudreault
c41c18bb8d HBASE-26875 RpcRetryingCallerImpl translateException ignores return value of recursive call (#4270)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-25 09:01:27 -07:00
Nick Dimiduk
b910b11731 HBASE-26531 Trace coprocessor exec endpoints
Trace table ExecService invocations as table operations. Ensure span relationships for both table
and master invocations.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-22 14:10:39 +01:00
Duo Zhang
6cef9e1602 HBASE-25465 Use javac --release option for supporting cross version compilation (#4164)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-12 18:30:40 +08:00
Duo Zhang
5844b53dea HBASE-26802 Backport the log4j2 changes to branch-2 (#4166)
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-hadoop-compat/pom.xml
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml
	hbase-shaded/hbase-shaded-client/pom.xml
	hbase-shaded/hbase-shaded-mapreduce/pom.xml
	hbase-shaded/hbase-shaded-testing-util/pom.xml
	hbase-shaded/pom.xml
	hbase-testing-util/pom.xml
2022-03-11 11:38:37 -08:00