Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,026 Commits 95 Branches 866 Tags
Commit Graph

8608 Commits

Author SHA1 Message Date
Tak Lon (Stephen) Wu
f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529) 
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-07-29 09:15:10 -07:00
Duo Zhang
dd22fecc2b
HBASE-26120 New replication gets stuck or data loss when multiwal groups more than 10 (#3528) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-27 08:14:15 -07:00
Duo Zhang
20a4aaedcc HBASE-26118 The HStore.commitFile and HStore.moveFileIntoPlace almost have the same logic (#3525) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-26 21:47:12 +08:00
Wei-Chiu Chuang
37c3f7b446
HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable (#3434) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4a3c7d73b04928ca0d63b7117186ddb9757997f2)
 2021-07-26 17:43:10 +08:00
Duo Zhang
be0a6cea40 HBASE-26119 Polish TestAsyncNonMetaRegionLocator (#3526) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-07-26 17:34:01 +08:00
Rushabh Shah
59e2825c93 HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory (#3504) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-23 09:33:43 -07:00
meiyi
729da5a6d3
HBASE-24734 RegionInfo#containsRange should support check meta table (#3496) (#3517) 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2021-07-22 18:04:24 +08:00
gkanade
8bc180c773
HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation (#3512) 
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-07-22 08:30:25 +05:30
Aman Poonia
cabf0dc51d
HBASE-25986 set default value of normalization enabled from hbase site (#3476) (#3372) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-07-19 15:26:52 +05:30
clarax
55d93fbf2d
HBASE-25973 Balancer should explain progress in a better way in log -… (#3483) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-18 13:04:33 -07:00
YutSean
fb01a6c276
HBASE-26083 In branch-2 & branch-1 L1 miss metric is always 0 when using CombinedBlockCache (#3473) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Reid Chan <reidchan@apache.org>
 2021-07-16 10:28:27 +08:00
Xiaolin Ha
692d384ec6
HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations (#3436) (#3486) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-14 16:30:46 +08:00
binlijin
12f2a16280
HBASE-26086 TestHRegionReplayEvents do not pass in branch-2 and throws NullPointerException (#3480) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-07-14 11:37:25 +08:00
Clara Xiong
dfc9ac8ffb HBASE-25739 TableSkewCostFunction need to use aggregated deviation - backport 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-07-13 08:37:29 -07:00
mokai
3294d32f46 HBASE-25700 Enhance znode parent validation when add_peer (#3458) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-07-11 21:15:57 +08:00
Duo Zhang
b86d2d4020 HBASE-26069 Remove HStore.compactRecentForTestingAssumingDefaultPolic… (#3462) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-07 21:57:59 +08:00
Duo Zhang
1ba03cae4b HBASE-26068 The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong (#3461) 
Signed-off-by: Yulin Niu <niuyulin@apache.org
 2021-07-07 14:39:03 +08:00
Duo Zhang
bde5d98338 HBASE-26065 StripeStoreFileManager does not need to throw IOException for most methods (#3459) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-07-05 22:19:17 +08:00
Viraj Jasani
342a478753
HBASE-22923 Consider minVersionToMoveSysTables while moving region and creating regionPlan (ADDENDUM) (#3455) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-03 22:44:46 +05:30
Wei-Chiu Chuang
db75ef265e
HBASE-26050 Remove the reflection used in FSUtils.isInSafeMode (#3445) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 82c44b49f54d92e8635d35117d6ae47a92938e30)
 2021-07-02 14:21:28 +08:00
Viraj Jasani
1381df4c80
HBASE-22923 min version of RegionServer to move system table regions (#3439) (#3438) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-01 16:24:14 +05:30
Viraj Jasani
75cd25386a
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade (#3417) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-01 15:20:38 +05:30
bsglz
b90fbb77c6 HBASE-26028 The view as json page shows exception when using TinyLfuBlockCache (#3420) 
(cherry picked from commit 147b030c1f46f35bbb72e2cf80ede85f7559bb5f)
 2021-07-01 10:27:24 +08:00
Duo Zhang
748e369629 HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death (#3430) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-06-30 11:46:09 +08:00
Duo Zhang
f827a4c5f8 HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 (#3440) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-30 08:27:17 +08:00
GeorryHuang
166becfd66
HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… (#3373) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:30:46 +08:00
GeorryHuang
c707a9ffff
HBASE-25914 Provide slow/large logs on RegionServer UI (#3321) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:27:06 +08:00
GeorryHuang
c7c5d56084 HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… (#3406) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:13:21 +08:00
YutSean
193a94e523
HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 (#3411) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-24 15:24:50 +08:00
Duo Zhang
9fe1781037 HBASE-26020 Split TestWALEntryStream.testDifferentCounts out (#3409) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-06-23 22:51:23 +08:00
litao
c81a5ab565
HBASE-25934 Add username for RegionScannerHolder (#3325) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-23 12:54:41 +05:30
lujiefsi
2ae69a883c HBASE-25877 Add access check for compactionSwitch (#3253) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 23:45:36 +08:00
belugabehr
bda7afe97d HBASE-25937: Clarify UnknownRegionException (#3330) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 22:44:05 +08:00
Duo Zhang
12d707c880 HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back 2021-06-21 23:58:42 +08:00
Viraj Jasani
41a68325c2
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache (#3215) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-21 12:51:59 +05:30
YutSean
029dc81b39
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid (#3403) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-21 15:00:50 +08:00
Duo Zhang
c800e6c825 HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 (#3376) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-20 16:38:13 +08:00
Bharath Vissapragada
3b58919246
HBASE-25998: Redo synchronization in SyncFuture (#3399) 
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch

- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb596421974717697b28d0856453245759c15)
 2021-06-18 10:37:10 -07:00
Bharath Vissapragada
836fd3c52a
HBASE-25984: Avoid premature reuse of sync futures in FSHLog (#3371) (#3392) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5a19bcfa98b3ccd9f7fb1fb933248c808676d91c)
 2021-06-17 12:27:23 -07:00
Duo Zhang
26c4e86ebd HBASE-25976 Implement a master based ReplicationTracker (#3390) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-17 18:38:43 +08:00
binlijin
4719a2078e HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… (#3380) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 14:39:47 +08:00
Toshihiro Suzuki
6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (#3386) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 13:55:21 +09:00
Duo Zhang
cb5f0dfe1e HBASE-25995 Change the method name for DoubleArrayCost.setCosts (#3381) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-15 11:55:54 +08:00
Andrew Purtell
a1d59bf1a5 HBASE-25994 Active WAL tailing fails when WAL value compression is enabled (#3377) 
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.

Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-14 17:18:53 -07:00
bsglz
88f031eeec HBASE-25967 The readRequestsCount does not calculate when the outResu… (#3351) 
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty

Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98c8b938ebd926e5bf8cfa69fd2f2a74)
 2021-06-10 10:24:18 +08:00
Xiaolin Ha
b6837295f3 HBASE-25981 JVM crash when displaying RegionServer UI (#3364) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-09 18:16:33 +08:00
Duo Zhang
e4dd4901a8 HBASE-25963 HBaseCluster should be marked as IA.Public (#3348) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-06 21:27:02 +08:00
meiyi
1267bdb1a4 HBASE-25929 RegionServer JVM crash when compaction (#3318) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-03 17:20:08 +08:00
Andrew Purtell
a4e8ee183e HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302) 
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
	hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 2021-06-01 12:41:15 -07:00
Bharath Vissapragada
d4285be5c1
HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3332) (#3345) 
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c77860fe430308a6bfd7dd8aebe1ad3510c)
 2021-06-01 08:04:12 -07:00