Commit Graph

2440 Commits

Author SHA1 Message Date
Josh Elser 1e74f08403 Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed"
This reverts commit 5b58d11c91.

I accidentally applied this as myself instead of as Bryan. Reverting it
to apply the patch properly.
2021-09-01 21:57:46 -04:00
Josh Elser 5b58d11c91 HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
2021-09-01 21:47:53 -04:00
Tak Lon (Stephen) Wu ab431fc8a9 HBASE-26140 Backport HBASE-25778 "The tracinig implementation for AsyncConnectionImpl.getHbck is incorrect" to branch-2 (#3631)
17/17 commits of HBASE-22120, original commit f36e153964

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu a2b8f7a3be HBASE-26134 Backport HBASE-25617 "Revisit the span names" to branch-2 (#3618)
11/17 commits of HBASE-22120, original commit 8d68f8cd1c

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu d0a53e3f29 HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 (#3608)
10/17 commits of HBASE-22120, original commit f6ff519dd0

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 8b7450f5d8 HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 (#3607)
9/17 commits of HBASE-22120, original commit bb8c4967f8

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 321dcd4e77 HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… (#3594)
7/17 commits of HBASE-22120, original commit 03e12bfa4a

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by:  Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 83a27389a0 HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 (#3589)
6/17 commits of HBASE-22120, original commit ae2c62ffaa

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by:  Peter Somogyi  <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 7ecf54f365 HBASE-26128 Backport HBASE-25454 "Add trace support for connection re… (#3561)
5/17 commits of HBASE-22120, original commit dcb78bd4bd and merged conflicts after rebasing on HBASE-26150 with commit 63d4970de4

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 58b5c007b9 HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… (#3556)
4/17 commits of HBASE-22120, original commit 805b2ae2ad

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 4dfc59c19e HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)
2/17 commits of HBASE-22120

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 665305cc3b HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Duo Zhang c08e877d6a HBASE-26235 We could start RegionServerTracker before becoming active master (#3645)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-08-30 12:22:41 +08:00
GeorryHuang 033ff757fd HBASE-26216 Move HRegionServer.abort(String) to Abortable as a default method (#3642)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-28 23:06:54 +08:00
Duo Zhang 931a7a5a61 HBASE-26180 Introduce a initial refresh interval for RpcConnectionRegistry (#3601)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-08-25 18:51:40 +08:00
Duo Zhang d8fb978bef HBASE-26182 Allow disabling refresh of connection registry endpoint (#3605)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-24 14:07:28 +08:00
Josh Elser 2728821e62
HBASE-26212 Expose configuration to enable/disable AuthUtil (#3619)
In some situations, a caller may know that it is properly managing the
Kerberos ticket to talk to HBase. In these situations, it's possible
that AuthUtil still tries to do renewals, but just fails repeatedly to
do so. Give a configuration flag for such clients to be able to tell
AuthUtil to simply stop trying.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-23 20:23:52 -04:00
Duo Zhang 6ad7eb88c3 HBASE-26172 Deprecated MasterRegistry (#3566)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-19 16:30:48 +08:00
Bo Cui 13c5cfa361 HBASE-24570 connection#close throws NPE 2021-08-16 17:05:50 -07:00
stack 243e861985 Revert "HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned (#3532)"
This reverts commit 8f16e34eb2.
2021-08-11 07:41:22 -07:00
Bryan Beaudreault 8f16e34eb2
HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned (#3532)
Signed-off-by: stack <stack@apache.org>
2021-08-10 20:38:06 -07:00
Tak Lon (Stephen) Wu c11a3e1b39 Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)"
This reverts commit f049301606.
2021-08-04 15:55:13 -07:00
Tak Lon (Stephen) Wu fffdcba5bb Revert "HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)"
This reverts commit ca096437d7.
2021-08-04 15:55:13 -07:00
Duo Zhang 620806e3fb HBASE-26150 Let region server also carry ClientMetaService (#3550)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-05 00:15:49 +08:00
Rahul Kumar 78fe9caa5c
HBASE-25651 NORMALIZER_TARGET_REGION_SIZE needs a unit in its name (#3450)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Baiqiang Zhao <ZhaoBQ>
2021-08-04 08:21:43 -07:00
Tak Lon (Stephen) Wu ca096437d7
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)
2/17 commits of HBASE-22120

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-08-02 11:33:19 -07:00
Tak Lon (Stephen) Wu f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-07-29 09:15:10 -07:00
stack 2c7b185431 Revert "HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFuture… (#3419)"
This reverts commit 1d6eb77ef8.
2021-07-22 21:06:27 -07:00
meiyi 729da5a6d3
HBASE-24734 RegionInfo#containsRange should support check meta table (#3496) (#3517)
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-07-22 18:04:24 +08:00
Rushabh Shah 253b83db57
HBASE-26088 Fix thread leaks in conn#getBufferedMutator(tableName) method call (#3506)
Co-authored-by: Rushabh Shah <rushabh.shah@rushabh-ltmflld.internal.salesforce.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@duboce.net>
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-07-21 17:32:44 +05:30
Aman Poonia cabf0dc51d
HBASE-25986 set default value of normalization enabled from hbase site (#3476) (#3372)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-07-19 15:26:52 +05:30
mokai 3294d32f46 HBASE-25700 Enhance znode parent validation when add_peer (#3458)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-07-11 21:15:57 +08:00
bsglz 1d6eb77ef8
HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFuture… (#3419)
* HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
2021-07-01 19:16:51 +08:00
Viraj Jasani 75cd25386a
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade (#3417)
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-01 15:20:38 +05:30
GeorryHuang c7c5d56084 HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… (#3406)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:13:21 +08:00
belugabehr bda7afe97d HBASE-25937: Clarify UnknownRegionException (#3330)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 22:44:05 +08:00
Andrew Purtell a4e8ee183e HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302)
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
	hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
2021-06-01 12:41:15 -07:00
Anoop Sam John 0ede56d24a HBASE-25903 ReadOnlyZKClient APIs - CompletableFuture.get() calls can cause threads to hang forver when ZK client create throws Non IOException (#3293)
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-05-31 11:39:14 +05:30
Bharath Vissapragada 59513e81b7
HBASE-25942: Get rid of null regioninfo in wrapped connection exceptions (#3337)
Reviewed-by: Tamanna Mehta
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 528f543e23)
2021-05-30 21:08:55 -07:00
mokai b4e8b91414
HBASE-25718 Backport "HBASE-25705 Convert proto to RSGroupInfo is costly" (#3109)
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-05-28 19:23:25 +08:00
Duo Zhang 8110b18ab0 HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-18 10:43:01 +08:00
Michael Stack 075545731a
HBASE-25876 Add retry if we fail to read all bytes of the protobuf magic marker (#3251)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2021-05-12 11:20:50 -07:00
GeorryHuang 63d49cb7ed
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) (#3245)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-09 00:09:23 +05:30
Andrew Purtell 01ce44abc4 HBASE-25756 Support alternate compression for major and minor compactions (#3142)
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
	hbase-shell/src/main/ruby/hbase/admin.rb
2021-04-23 16:02:41 -07:00
ZhiChen 46a106014e HBASE-25762 Improvement for some debug-logging guards (#3145)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-13 23:25:03 +08:00
Andrew Purtell 3f1269e6c6 HBASE-25750 Upgrade RpcControllerFactory and HBaseRpcController from Private to LimitedPrivate(COPROC,PHOENIX) (#3136)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-04-08 17:54:03 -07:00
stack 96a177fdc4 HBASE-25735 Add target Region to connection exceptions
Restore API for Phoenix (though it shouldn't be using
Private classes).
2021-04-08 13:39:23 -07:00
stack 50bd11a2e3 HBASE-25735 Add target Region to connection exceptions
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-04-06 18:06:42 -07:00
Jan Hentschel 18882d6f9a
HBASE-24305 Prepare deprecations in ServerName (#1666) (#3121)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-04-05 22:10:04 +02:00
stack b051ad5594 Revert "HBASE-25713 Make an hbase-wal module"
This reverts commit e58f1b24a6.
2021-03-31 16:15:50 -07:00
stack e58f1b24a6 HBASE-25713 Make an hbase-wal module
First extract an hbase-coprocessor module used by hbase-client, hbase-server.
This is prerequisite to extracting an hbase-wal module.

M hbase-common/src/main/java/org/apache/hadoop/hbase/Abortable.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/DoNotRetryIOException.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/SortedList.java
 Move to hbase-common. Its a generic Interface. Need by

M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/CoprocessorEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContext.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContextImpl.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ReadOnlyConfiguration.java
 Move to hbase-coprocessor.

M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/BigDecimalColumnInterpreter.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/DoubleColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/LongColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/ColumnInterpreter.java
 Moved to hbase-endpoint where they are used.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
 Include region name when toString'd.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCoprocessorHost.java
 Include WAL name when toString'd.
M hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 Add utility used in testing here from CoprocessorHost.
2021-03-31 16:14:45 -07:00
Toshihiro Suzuki 1d3ea38f1e
HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3107)
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 23:04:29 +09:00
Toshihiro Suzuki 2b5c2c36d9
HBASE-25678 Support nonce operations for Increment/Append in RowMutations and CheckAndMutate (#3073)
Signed-off-by: stack <stack@apache.org>
2021-03-21 22:47:53 +09:00
Catalin Luca c9b81c3f82 HBASE-25674 - RegionInfo.parseFrom(DataInputStream) sometimes fails to read the protobuf magic marker (#3062)
Co-authored-by: Catalin Luca <luca@adobe.com>
Signed-off-by: stack <stack@apache.org>
2021-03-18 11:56:34 -07:00
bitterfox a61e9b963d HBASE-25665 Option to use hostname instead of canonical hostname for secure HBase cluster connection (#3051) 2021-03-16 21:05:23 -07:00
Michael Stack 763de9a085
HBASE-25561 Added ignored test for async connection that runs retries just so can check how long it takes and that retrying is happening (#2942) 2021-03-15 14:14:41 -07:00
shahrs87 bb4d8b9cf7 HBASE-25622 Result#compareResults should compare tags. (#3026)
Signed-off-by: stack <stack@apache.org>
2021-03-11 21:52:11 -08:00
Peter Somogyi af8dc2e4df HBASE-25587 [hbck2] Schedule SCP for all unknown servers (#2978)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-11 17:22:08 +01:00
Wellington Ramos Chevreuil 7adc00044c HBASE-25548 Optionally allow snapshots to preserve cluster's max file… (#2923)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 373dc7788d)
2021-03-10 10:35:58 +00:00
Anoop Sam John f2c52e4176 HBASE-25644 Scan#setSmall blindly sets ReadType as PREAD
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2021-03-08 13:10:52 +05:30
Mohammad Arshad 0487760829
HBASE-25492: Create table with RSGroup (#2883)
Added new interface in TableDescriptor which allows user to define RSGroup name while creating or modifying a table.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2021-03-06 10:44:19 +05:30
lidingshengHHU acf2bd9c2d HBASE-25421 There is no limit on the column length when creating a table (#2796)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-03-03 14:43:37 +08:00
XinSun 328ff8c05a HBASE-25590 Bulkload replication HFileRefs cannot be cleared in some cases where set exclude-namespace/exclude-table-cfs (#2969)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-02-26 09:58:05 +08:00
Toshihiro Suzuki cfbae4d3a3 HBASE-25575 Should validate Puts in RowMutations
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-02-22 11:02:13 +09:00
Baiqiang Zhao 9a6bd5b6f1
HBASE-25519 BLOCKSIZE needs to support pretty print (#2944) 2021-02-09 07:27:27 -08:00
Baiqiang Zhao 3b43df68e6
HBASE-25533 The metadata of the table and family should not be an empty string (#2914) (#2906)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
2021-01-29 21:00:30 +05:30
Baiqiang Zhao c5674e21dd
HBASE-25439 Add BYTE unit in PrettyPrinter.Unit (#2840)
Signed-off-by: stack <stack@apache.org>
2021-01-14 21:47:01 -08:00
Viraj Jasani 0788547fea
HBASE-25474 : Bump HBase version on branch-2 (#2871)
Signed-off-by: stack <stack@apache.org>
2021-01-12 10:20:22 +05:30
Duo Zhang 81d17d063c HBASE-25457 Possible race in AsyncConnectionImpl between getChoreServ… (#2839)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-04 23:54:52 +08:00
Duo Zhang 15d27450c2 HBASE-25451 Upgrade commons-io to 2.8.0 (#2825)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-12-31 17:11:49 +08:00
Duo Zhang 9325a775fe HBASE-25452 Use MatcherAssert.assertThat instead of org.junit.Assert.assertThat (#2826)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-12-30 10:03:39 +08:00
stack b07b400b62 HBASE-25425 Some notes on RawCell 2020-12-21 08:13:42 -08:00
scotthunt befa76f55a HBASE-25084 Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-12-21 08:48:14 +00:00
Duo Zhang 1385fb3560 HBASE-25420 Some minor improvements in rpc implementation (#2792)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
2020-12-20 11:30:16 +08:00
huaxiangsun 8159fc8dbe
HBASE-25293 Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side. (#2768) (#2786)
Signed-off-by: stack <stack@apache.org>
2020-12-18 09:25:04 -08:00
Sandeep Pal aa5a477426
HBASE-25383: Ability to update and remove peer base config
Closes #2778

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-18 14:20:36 +05:30
huaxiangsun 22dd0545b9
HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName (#2759)
Signed-off-by: stack <stack@apache.org>
2020-12-17 15:21:30 -08:00
shahrs87 9b4be47c24
HBASE-25246 Backup/Restore hbase cell tags (#2766)
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-16 18:59:19 +05:30
Duo Zhang ec47e46c7e HBASE-25336 Use Address instead of InetSocketAddress in RpcClient implementation (#2716)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-12-07 21:50:16 +08:00
stack 7bb3ab7687 HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster 2020-12-05 14:21:52 -08:00
Andrew Purtell ea331a66be HBASE-25292 Improve InetSocketAddress usage discipline (#2669)
Network identities should be bound late. Remote addresses should be
resolved at the last possible moment, just before connect(). Network
identity mappings can change, so our code should not inappropriately
cache them. Otherwise we might miss a change and fail to operate normally.

Revert "HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors"
Removes hbase.resolve.hostnames.on.failure and related code. We always
resolve hostnames, as late as possible.

Preserve InetSocketAddress caching per RPC connection. Avoids potential
lookups per Call.

Replace InetSocketAddress with Address where used as a map key. If we want
to key by hostname and/or resolved address we should be explicit about it.
Using Address chooses mapping by hostname and port only.

Add metrics for potential nameservice resolution attempts, whenever an
InetSocketAddress is instantiated for connect; and metrics for failed
resolution, whenever InetSocketAddress#isUnresolved on the new instance
is true.

* Use ServerName directly to build a stub key

* Resolve and cache ISA on a RpcChannel as late as possible, at first call

* Remove now invalid unit test TestCIBadHostname

We resolve DNS at the latest possible time, at first call, and do not
resolve hostnames for creating stubs at all, so this unit test cannot
work now.

Reviewed-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-04 10:19:02 -08:00
Viraj Jasani a95c0c6537
Revert "HBASE-25246 Backup/Restore hbase cell tags"
This reverts commit 34721c42ec.
2020-12-03 21:17:11 +05:30
shahrs87 34721c42ec
HBASE-25246 Backup/Restore hbase cell tags
Closes #2706

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-02 21:30:46 +05:30
clarax 4c8fa47b28
HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2655)
* HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)


Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-11-30 22:51:57 -08:00
Balazs Meszaros b75b772bc6
HBASE-25307 ThreadLocal pooling leads to NullPointerException (#2709)
* PoolMap does not discard any elements anymore. If an element is put,
it always stores it. The reason: it stores expensive resources (rpc
connections) which would lead to resource leak if we simple discard it.
RpcClients can reference netty ByteBufs which are reference counted.
Resource cleanup is done by AbstractRpcClient.cleanupIdleConnections().
* PoolMap does not implement Map interface anymore, so ensuring
thread-safety has become easier. Put method is replaced with getOrCreate().
* ThreadLocalPool doesn't use ThreadLocal class anymore. It stores
resources on thread basis, but it doesn't remove values when a thread
exits. Again, proper cleanup is done by cleanupIdleConnections().

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
2020-11-30 12:36:03 +01:00
Toshihiro Suzuki 3775464981
HBASE-25242 Add Increment/Append support to RowMutations (#2711)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-11-27 03:53:19 +09:00
Mate Szalay-Beko 1dbfe96d69 HBASE-25263 Various improvements of column family encryption
This PR is a follow-up of HBASE-25181 (#2539), where several issues were
discussed on the PR:

1. Currently we use PBKDF2WithHmacSHA1 key generation algorithm to generate a
secret key for HFile / WalFile encryption, when the user is defining a string
encryption key in the hbase shell. This algorithm is not secure enough and
not allowed in certain environments (e.g. on FIPS compliant clusters). We are
changing it to PBKDF2WithHmacSHA384. It will not break backward-compatibility,
as even the tables created by the shell using the new algorithm will be able
to load (e.g. during bulkload / replication) the HFiles serialized with the
key generated by an old algorithm, as the HFiles themselves already contain
the key necessary for their decryption.

Smaller issues fixed by this commit:

2. Improve the documentation e.g. with the changes introduced by HBASE-25181
and also by some points discussed on the Jira ticket of HBASE-25263.

3. In EncryptionUtil.createEncryptionContext the various encryption config
checks should throw IllegalStateExceptions instead of RuntimeExceptions.

4. Test cases in TestEncryptionTest.java should be broken down into smaller
tests.

5. TestEncryptionDisabled.java should use ExpectedException JUnit rule to
validate exceptions.

closes #2676

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-11-26 16:57:12 +01:00
niuyulin 751b4f040e HBASE-24872 refactor valueOf PoolType (#2250)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-11-26 12:50:59 +01:00
Jonathan M Hsieh 35a2875dc1 HBASE-24827 BackPort HBASE-11554 Remove Reusable poolmap Rpc client type. (#2208)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>
Signed-off-by: stack <stack@apache.org>
2020-11-26 11:22:40 +01:00
Andrew Purtell 1b6399a480
HBASE-24640 [branch-2] Purge use of VisibleForTesting (#2696)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-11-25 14:15:32 -08:00
stack 73086f25e6 HBASE-25291 Document how to enable the meta replica load balance mode for the client and clean up around hbase:meta read replicas
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSelector.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSelectorFactory.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaLoadBalanceSimpleSelector.java
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CatalogReplicaMode.java
 Make the leading license comment opener align on all these new
 files.... one star rather than two.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/CatalogReplicationSource.java
 Add some comment better situating the catalogreplicationsource
 specialization.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
 Add comment how this differs from replicationsource factory and why no
 need to keep offset up in zk.

src/main/asciidoc/_chapters/architecture.adoc
 Clean up the read replica of META doc. Introduce the new read replica
 liveness on META and the new 'LoadBalance' feature (with warnings).
 General edit.
2020-11-20 15:49:00 -08:00
Huaxiang Sun 920cf9faf4 HBASE-25126 Add load balance logic in hbase-client to distribute read load over meta replica regions
It adds load balance support for meta lookup in AsyncTableRegionLocator.
The existing meta replica mode is renamed as "HedgedRead", client sends scan request to the primary meta replica region first,
if response is not back within a configured amount of time, it will send scan requests to all meta replica regions and
take the first response. On top of the existing mode, a new mode "LoadBalance" is introduced. In this mode, client first
choose a meta replica region to send scan request. If the response is stale, it may send the request to another meta replica region or
the primary region. In this mode, meta scan requests are load balanced across all replica regions with the primary mode as
the ultimate source of truth.

Two new config knobs are added:

1. hbase.locator.meta.replicas.mode
   Valid options are "None", "HedgedRead" and "LoadBalance", they are case insensitive. The default mode is "None".

2. hbase.locator.meta.replicas.mode.loadbalance.selector
   The load balance alogrithm to select a meta replica to send the requests.
   Only org.apache.hadoop.hbase.client.CatalogReplicaLoadBalanceReplicaSimpleSelector.class
   is supported for now, which is the default as well. The algorithm works as follows:
      a. Clients select a randome meta replica region to send the requests if there is no entry for the location in the stale
         location cache.
      b. If the location from one meta replica region is stale, a stale entry will be created in the statle location cache
         for the region.
      c. Clients select the primary meta region if the location is in the stale location cache.
      d. The stale location cache entries time out in 3 seconds.

If there is no "hbase.locator.meta.replicas.mode" configured, it will check the config knob "hbase.meta.replicas.use".
If "hbase.meta.replicas.use" is configured, the mode will be set to "HedgedRead".

For branch-2 support, it introduces support for ReversedScan over a specific non-default replica region, this is mainly
for load balance meta scan among all its replica regions.

Remove ConnectionImplementation#setUseMetaReplicas()
2020-11-20 10:45:24 -08:00
Michael Stack 47ad207664 HBASE-25280 [meta replicas] ArrayIndexOutOfBoundsException in ZKConnectionRegistry (#2652)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-11-20 00:31:43 -08:00
Duo Zhang b19df076a4
HBASE-25272 Support scan on a specific replica (#2645)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2020-11-12 21:25:53 +08:00
Mate Szalay-Beko 9fe62c3ec6
HBASE-25181 Add options for disabling column family encryption and choosing hash algorithm for wrapped encryption keys.
Prior to this patch hbase always used the MD5 hash algorithm to store a hash for encryption keys.
This hash is needed to verify the secret key of the subject. (e.g. making
sure that the same secrey key is used during encrypted HFile read and write).
The MD5 algorithm is considered weak, and can not be used in some
(e.g. FIPS compliant) clusters.

In this patch we:
- add a config parameter to globally enable/disable column family encryption (def enabled)
- introduce a backward compatible way of specifying the hash algorithm.
  This enable us to use newer and more secure hash algorithms like SHA-384
  or SHA-512 (which are FIPS compliant).
- add a config parameter to fail if an hfile is encountered that uses a
  different hash algorithm than the one currently configured to ease validation after
  migrating key hash algorithms (def disabled)

Closes #2539

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
(cherry picked from commit 6a5c928539)
2020-11-09 14:15:22 -06:00
Michael Stack df0009d879
HBASE-25238 Upgrading HBase from 2.2.0 to 2.3.x fails because of “Message missing required fields: state” (#2625)
Make protobuf fields add post-2.0.0 release marked 'required' instead
be 'optional' so migrations from 2.0.x to 2.1+ or 2.2+ succeeds.
2020-11-05 08:36:55 -08:00
niuyulin 770098128a HBASE-25210 RegionInfo.isOffline is now a duplication with RegionInfo.isSplit (#2580)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-11-04 08:11:00 -08:00
bsglz 3a45fc17e5
HBASE-24967 The table.jsp cost long time to load if the table include closed regions (#2613) 2020-11-01 21:11:15 +08:00
niuyulin bfe92ea53b HBASE-25201 YouAreDeadException should be moved to hbase-server module (#2581)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 15:02:30 +08:00
niuyulin 1bf68514ea HBASE-25176 MasterStoppedException should be moved to hbase-client module (#2538)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 15:02:20 +08:00