Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,748 Commits 95 Branches 866 Tags
Commit Graph

8428 Commits

Author SHA1 Message Date
litao
c81a5ab565
HBASE-25934 Add username for RegionScannerHolder (#3325) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-23 12:54:41 +05:30
lujiefsi
2ae69a883c HBASE-25877 Add access check for compactionSwitch (#3253) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 23:45:36 +08:00
belugabehr
bda7afe97d HBASE-25937: Clarify UnknownRegionException (#3330) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 22:44:05 +08:00
Duo Zhang
12d707c880 HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back 2021-06-21 23:58:42 +08:00
Viraj Jasani
41a68325c2
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache (#3215) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-21 12:51:59 +05:30
YutSean
029dc81b39
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid (#3403) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-21 15:00:50 +08:00
Duo Zhang
c800e6c825 HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 (#3376) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-20 16:38:13 +08:00
Bharath Vissapragada
3b58919246
HBASE-25998: Redo synchronization in SyncFuture (#3399) 
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch

- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb596421974717697b28d0856453245759c15)
 2021-06-18 10:37:10 -07:00
Bharath Vissapragada
836fd3c52a
HBASE-25984: Avoid premature reuse of sync futures in FSHLog (#3371) (#3392) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5a19bcfa98b3ccd9f7fb1fb933248c808676d91c)
 2021-06-17 12:27:23 -07:00
Duo Zhang
26c4e86ebd HBASE-25976 Implement a master based ReplicationTracker (#3390) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-17 18:38:43 +08:00
binlijin
4719a2078e HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… (#3380) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 14:39:47 +08:00
Toshihiro Suzuki
6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (#3386) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 13:55:21 +09:00
Duo Zhang
cb5f0dfe1e HBASE-25995 Change the method name for DoubleArrayCost.setCosts (#3381) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-15 11:55:54 +08:00
Andrew Purtell
a1d59bf1a5 HBASE-25994 Active WAL tailing fails when WAL value compression is enabled (#3377) 
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.

Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-14 17:18:53 -07:00
bsglz
88f031eeec HBASE-25967 The readRequestsCount does not calculate when the outResu… (#3351) 
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty

Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98c8b938ebd926e5bf8cfa69fd2f2a74)
 2021-06-10 10:24:18 +08:00
Xiaolin Ha
b6837295f3 HBASE-25981 JVM crash when displaying RegionServer UI (#3364) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-09 18:16:33 +08:00
Duo Zhang
e4dd4901a8 HBASE-25963 HBaseCluster should be marked as IA.Public (#3348) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-06 21:27:02 +08:00
meiyi
1267bdb1a4 HBASE-25929 RegionServer JVM crash when compaction (#3318) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-03 17:20:08 +08:00
Andrew Purtell
a4e8ee183e HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302) 
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
	hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 2021-06-01 12:41:15 -07:00
Bharath Vissapragada
d4285be5c1
HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3332) (#3345) 
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c77860fe430308a6bfd7dd8aebe1ad3510c)
 2021-06-01 08:04:12 -07:00
Michael Stack
1a45b743a0 HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO (#3334) 
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-29 09:04:08 -07:00
Sandeep Pal
5157521213
HBASE-25927: Fix the log messages by not stringifying the exceptions in log (#3338) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 9a2027bf714fa9787733c442561ec1ac2b4a0153)
 2021-05-28 21:34:14 -07:00
Duo Zhang
2a84fd40d3 HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one (#3329) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-29 11:10:52 +08:00
Baiqiang Zhao
bbe9558040
HBASE-25745 Deprecate/Rename config hbase.normalizer.min.region.count to hbase.normalizer.merge.min.region.count 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-28 13:10:20 -07:00
Victor
5aa39a975c
HBASE-25910 - Fix port assignment test (#3308) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-28 20:53:20 +05:30
Wellington Ramos Chevreuil
cad9ae386a HBASE-25933 Log trace raw exception, instead of cause message in NettyRpcServerRequestDecoder (#3323) 
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2021-05-27 20:03:17 +01:00
Duo Zhang
3cf0678aee HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code (#3313) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-27 16:25:01 +08:00
Rushabh Shah
e265eccf20
HBASE-25924 Re-compute size of WAL file while removing from WALEntryStream (#3315) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-05-26 10:42:03 -07:00
Baiqiang Zhao
563ebc214a
HBASE-25534 Honor TableDescriptor settings earlier in normalization (#2917) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-25 08:24:05 -07:00
Duo Zhang
2cb6cc8a6c HBASE-25894 Improve the performance for region load and region count related cost functions (#3276) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-25 18:07:33 +08:00
GeorryHuang
8754e88fa7
HBASE-25906 UI of master-status to show recent history of balancer desicion (#3303) (#3296) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-25 12:39:47 +05:30
Baiqiang Zhao
073f23e2bd HBASE-25513 When the table is turned on normalize, the first region may not be merged even the size is 0 (#2887) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-24 17:27:06 -07:00
Anoop Sam John
1649013b99 HBASE-25898 RS getting aborted due to NPE in Replication WALEntryStream (#3292) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-05-24 23:44:32 +05:30
Xiaolin Ha
76d56bc42e HBASE-25899 Improve efficiency of SnapshotHFileCleaner (#3280) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-24 22:18:43 +08:00
Duo Zhang
1c1acee2fb HBASE-25873 Refactor and cleanup the code for CostFunction (#3274) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-24 22:13:01 +08:00
Xiaolin Ha
16f865f65d HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3160) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-22 22:20:47 +08:00
caoliqing
a19dee1eed HBASE-25892: 'False' should be 'True' in auditlog of listLabels (#3273) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-22 17:25:43 +08:00
Sandeep Pal
926c1132c0
HBASE-25848: Add flexibility to backup replication in case replication filter throws an exception 
Signed-off-by: Xu Cang <xcang@salesforce.com>
 2021-05-21 12:14:35 -07:00
Andrew Purtell
2d591ab3c4 HBASE-25869 WAL value compression (#3244) 
WAL storage can be expensive, especially if the cell values
represented in the edits are large, consisting of blobs or
significant lengths of text. Such WALs might need to be kept around
for a fairly long time to satisfy replication constraints on a space
limited (or space-contended) filesystem.

We have a custom dictionary compression scheme for cell metadata that
is engaged when WAL compression is enabled in site configuration.
This is fine for that application, where we can expect the universe
of values and their lengths in the custom dictionaries to be
constrained. For arbitrary cell values it is better to use one of the
available compression codecs, which are suitable for arbitrary albeit
compressible data.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-21 11:53:36 -07:00
Rushabh Shah
7b374b7c76
HBASE-25827 Per Cell TTL tags get duplicated with increments causing tags length overflow (#3290)(#3210) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-21 22:58:32 +05:30
Duo Zhang
d78c164429 HBASE-25897 TestRetainAssignmentOnRestart is flaky after HBASE-25032 (#3281) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-05-20 21:18:09 +08:00
GeorryHuang
3040be1438
HBASE-25791 UI of master-status to show a recent history of that why balancer was rejected to run (#3278) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-20 15:00:00 +08:00
Nick Dimiduk
aab6e1d0ff
HBASE-25861 Correct the usage of Configuration#addDeprecation (#3249) 
Co-authored-by: Baiqiang Zhao <zbq.dean@gmail.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-18 06:53:31 -07:00
Duo Zhang
8110b18ab0 HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-18 10:43:01 +08:00
Pankaj
1a8d3c3b6b HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager (#3250) 
* HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager's start & retrievePassword method

Signed-off-by: stack <stack@apache.com>
(cherry picked from commit 2126ec94f0032220fcccdca463f0f185e3653ace)
 2021-05-17 12:21:47 +05:30
Duo Zhang
439fd4aff2 HBASE-25872 Add documentation for LoadBalancer about synchronization (#3267) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-05-17 10:20:48 +08:00
Duo Zhang
d5d972f92e HBASE-25883 The regionFinder and rackManager fields in BaseLoadBalancer should be volatile (#3262) 
Signed-off-by: meiyi <myimeiyi@gmail.com>
 2021-05-17 09:54:40 +08:00
Viraj Jasani
1bb16fea5b
HBASE-25884 Return empty records for disabled balancer in-memory queue (#3263) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
 2021-05-14 12:57:24 +05:30
Duo Zhang
72d7641fb5
HBASE-25852 Move all the intialization work of LoadBalancer implementation to initialize method (#3248) (#3258) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-13 14:19:03 +08:00
Michael Stack
075545731a
HBASE-25876 Add retry if we fail to read all bytes of the protobuf magic marker (#3251) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2021-05-12 11:20:50 -07:00