748e369629
HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death ( #3430 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-06-30 11:46:09 +08:00
f827a4c5f8
HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 ( #3440 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-30 08:27:17 +08:00
166becfd66
HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… ( #3373 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:30:46 +08:00
c707a9ffff
HBASE-25914 Provide slow/large logs on RegionServer UI ( #3321 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:27:06 +08:00
c7c5d56084
HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… ( #3406 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:13:21 +08:00
84ae6185b8
HBASE-26025 Add a flag to mark if the IOError can be solved by retry in thrift IOError ( #3416 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-25 16:44:22 +08:00
7e6b66c867
HBASE-26012 Improve logging and dequeue logic in DelayQueue ( #3397 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-24 15:28:36 +05:30
193a94e523
HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 ( #3411 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-24 15:24:50 +08:00
a7cbc064fa
HBASE-25966 Fix typo in NOTICE.vm
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2021-06-23 09:48:55 -07:00
9fe1781037
HBASE-26020 Split TestWALEntryStream.testDifferentCounts out ( #3409 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-06-23 22:51:23 +08:00
e3eb760864
HBASE-26019 Remove the reflection and call Configuration.getPassword() directly. ( #3408 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Wellington Chevreuil <wchevreuil@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
2021-06-23 13:25:42 +02:00
1978faee0e
Revert "Remove the reflection and call Configuration.getPassword() directly. ( #3408 )"
...
This reverts commit 669ff7be82
2021-06-23 13:25:10 +02:00
c81a5ab565
HBASE-25934 Add username for RegionScannerHolder ( #3325 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-06-23 12:54:41 +05:30
669ff7be82
Remove the reflection and call Configuration.getPassword() directly. ( #3408 )
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Wellington Chevreuil <wchevreuil@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
(cherry picked from commit 9a324bd4d0
2021-06-23 09:17:22 +08:00
2ae69a883c
HBASE-25877 Add access check for compactionSwitch ( #3253 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 23:45:36 +08:00
bda7afe97d
HBASE-25937: Clarify UnknownRegionException ( #3330 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 22:44:05 +08:00
12d707c880
HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back
2021-06-21 23:58:42 +08:00
41a68325c2
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache ( #3215 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-21 12:51:59 +05:30
029dc81b39
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid ( #3403 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-21 15:00:50 +08:00
c800e6c825
HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 ( #3376 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-20 16:38:13 +08:00
3b58919246
HBASE-25998: Redo synchronization in SyncFuture ( #3399 )
...
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch
- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb59642
2021-06-18 10:37:10 -07:00
836fd3c52a
HBASE-25984: Avoid premature reuse of sync futures in FSHLog ( #3371 ) ( #3392 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5a19bcfa98
2021-06-17 12:27:23 -07:00
26c4e86ebd
HBASE-25976 Implement a master based ReplicationTracker ( #3390 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-17 18:38:43 +08:00
a238e79e0f
HBASE-26000 Optimize the display of ZK dump in the master web UI ( #3383 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 14:15:32 +01:00
4719a2078e
HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… ( #3380 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 14:39:47 +08:00
6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update ( #3386 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 13:55:21 +09:00
cb5f0dfe1e
HBASE-25995 Change the method name for DoubleArrayCost.setCosts ( #3381 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-15 11:55:54 +08:00
a1d59bf1a5
HBASE-25994 Active WAL tailing fails when WAL value compression is enabled ( #3377 )
...
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.
Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-14 17:18:53 -07:00
ef9149bf78
HBASE-25989 FanOutOneBlockAsyncDFSOutput using shaded protobuf in hdfs 3.3+ ( #3368 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-12 08:29:06 -06:00
1542b50b00
HBASE-25993 Make excluded SSL cipher suites configurable for all Web UIs ( #3375 )
...
When starting a jetty http server, one can explicitly exclude certain (unsecure)
SSL cipher suites. This can be especially important, when the HBase cluster
needs to be compliant with security regulations (e.g. FIPS).
Currently it is possible to set the excluded ciphers for the ThriftServer
("hbase.thrift.ssl.exclude.cipher.suites") or for the RestServer
("hbase.rest.ssl.exclude.cipher.suites"), but one can not configure it for the
regular InfoServer started by e.g. the master or region servers.
In this commit I want to introduce a new configuration
"ssl.server.exclude.cipher.list" to configure the excluded cipher suites for the
http server started by the InfoServer. This parameter has the same name and will
work in the same way, as it was already implemented in hadoop (e.g. for hdfs/yarn).
See: HADOOP-12668, HADOOP-14341
Co-authored-by: Mate Szalay-Beko <symat@apache.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-06-10 16:48:51 +02:00
88f031eeec
HBASE-25967 The readRequestsCount does not calculate when the outResu… ( #3351 )
...
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty
Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98
2021-06-10 10:24:18 +08:00
d22bd9057e
HBASE-25987 Make SSL keystore type configurable for HBase ThriftServer ( #3367 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
(cherry picked from commit 7f7a293cb5
2021-06-09 16:57:01 +01:00
b6837295f3
HBASE-25981 JVM crash when displaying RegionServer UI ( #3364 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-09 18:16:33 +08:00
3d6e73d4e3
HBASE-25969 Purge netty-all transitive includes ( #3353 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Xiaolin Ha
2021-06-08 08:33:29 -07:00
2a45ce1434
HBASE-25918 Upgrade hbase-thirdparty dependency to 3.5.1 ( #3317 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit be14605898
2021-06-08 11:10:29 +05:30
e4dd4901a8
HBASE-25963 HBaseCluster should be marked as IA.Public ( #3348 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-06 21:27:02 +08:00
51a6aed72e
HBASE-25930 Thrift does not support requests in Kerberos environment ( #3326 )
...
Co-authored-by: sunhaoning <sunhaoning@qianxin.com>
Signed-off-by: Istvan Toth <stoty@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2021-06-04 08:17:17 +02:00
1267bdb1a4
HBASE-25929 RegionServer JVM crash when compaction ( #3318 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-03 17:20:08 +08:00
a4e8ee183e
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime ( #3302 )
...
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
2021-06-01 12:41:15 -07:00
d4285be5c1
HBASE-25932: Ensure replication reads the trailer bytes from WAL. ( #3332 ) ( #3345 )
...
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).
While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c7786
2021-06-01 08:04:12 -07:00
0ede56d24a
HBASE-25903 ReadOnlyZKClient APIs - CompletableFuture.get() calls can cause threads to hang forver when ZK client create throws Non IOException ( #3293 )
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-05-31 11:39:14 +05:30
920a9568b4
HBASE-25948 Remove deprecated ZK command 'rmr' in hbase-cleanup.sh ( #3343 )
...
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit bd4c4cec1f
2021-05-31 11:06:26 +05:30
9beb278d8b
HBASE-25941 TestRESTServerSSL fails because of jdk bug ( #3335 )
...
Signed-off-by: Wei-Chiu Chuang weichiu@apache.org
2021-05-30 21:33:34 -07:00
59513e81b7
HBASE-25942: Get rid of null regioninfo in wrapped connection exceptions ( #3337 )
...
Reviewed-by: Tamanna Mehta
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 528f543e23
2021-05-30 21:08:55 -07:00
1a45b743a0
HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO ( #3334 )
...
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-29 09:04:08 -07:00
5157521213
HBASE-25927: Fix the log messages by not stringifying the exceptions in log ( #3338 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 9a2027bf71
2021-05-28 21:34:14 -07:00
2a84fd40d3
HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one ( #3329 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-29 11:10:52 +08:00
fada9884a4
HBASE-25908 Exclude jakarta.activation-api ( #3299 )
...
Reapply of a revert because commit message was in wrong format
Signed-off-by: stack <stack@apache.org>
2021-05-28 13:27:13 -07:00
5ce6005550
Revert "Subject: [PATCH] HBASE-25908 Exclude jakarta.activation-api ( #3299 )"
...
This reverts commit 1e01c94ba3
2021-05-28 13:27:13 -07:00
bbe9558040
HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.count` to `hbase.normalizer.merge.min.region.count`
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-28 13:10:20 -07:00
Duo Zhang
GeorryHuang
YutSean
Viraj Jasani
YutSean
Nick Dimiduk
Wei-Chiu Chuang
Peter Somogyi
litao
lujiefsi
belugabehr
Bharath Vissapragada
binlijin
Toshihiro Suzuki
Andrew Purtell
Michael Stack
Mate Szalay-Beko
bsglz
Xiaolin Ha
Pankaj
Haoning Sun
meiyi
Anoop Sam John
mokai
Sandeep Pal
stack
Baiqiang Zhao