a238e79e0f
HBASE-26000 Optimize the display of ZK dump in the master web UI ( #3383 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 14:15:32 +01:00
4719a2078e
HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… ( #3380 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 14:39:47 +08:00
6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update ( #3386 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 13:55:21 +09:00
cb5f0dfe1e
HBASE-25995 Change the method name for DoubleArrayCost.setCosts ( #3381 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-15 11:55:54 +08:00
a1d59bf1a5
HBASE-25994 Active WAL tailing fails when WAL value compression is enabled ( #3377 )
...
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.
Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-14 17:18:53 -07:00
ef9149bf78
HBASE-25989 FanOutOneBlockAsyncDFSOutput using shaded protobuf in hdfs 3.3+ ( #3368 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-12 08:29:06 -06:00
1542b50b00
HBASE-25993 Make excluded SSL cipher suites configurable for all Web UIs ( #3375 )
...
When starting a jetty http server, one can explicitly exclude certain (unsecure)
SSL cipher suites. This can be especially important, when the HBase cluster
needs to be compliant with security regulations (e.g. FIPS).
Currently it is possible to set the excluded ciphers for the ThriftServer
("hbase.thrift.ssl.exclude.cipher.suites") or for the RestServer
("hbase.rest.ssl.exclude.cipher.suites"), but one can not configure it for the
regular InfoServer started by e.g. the master or region servers.
In this commit I want to introduce a new configuration
"ssl.server.exclude.cipher.list" to configure the excluded cipher suites for the
http server started by the InfoServer. This parameter has the same name and will
work in the same way, as it was already implemented in hadoop (e.g. for hdfs/yarn).
See: HADOOP-12668, HADOOP-14341
Co-authored-by: Mate Szalay-Beko <symat@apache.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-06-10 16:48:51 +02:00
88f031eeec
HBASE-25967 The readRequestsCount does not calculate when the outResu… ( #3351 )
...
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty
Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98
2021-06-10 10:24:18 +08:00
d22bd9057e
HBASE-25987 Make SSL keystore type configurable for HBase ThriftServer ( #3367 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
(cherry picked from commit 7f7a293cb5
2021-06-09 16:57:01 +01:00
b6837295f3
HBASE-25981 JVM crash when displaying RegionServer UI ( #3364 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-09 18:16:33 +08:00
3d6e73d4e3
HBASE-25969 Purge netty-all transitive includes ( #3353 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Xiaolin Ha
2021-06-08 08:33:29 -07:00
2a45ce1434
HBASE-25918 Upgrade hbase-thirdparty dependency to 3.5.1 ( #3317 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit be14605898
2021-06-08 11:10:29 +05:30
e4dd4901a8
HBASE-25963 HBaseCluster should be marked as IA.Public ( #3348 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-06 21:27:02 +08:00
51a6aed72e
HBASE-25930 Thrift does not support requests in Kerberos environment ( #3326 )
...
Co-authored-by: sunhaoning <sunhaoning@qianxin.com>
Signed-off-by: Istvan Toth <stoty@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2021-06-04 08:17:17 +02:00
1267bdb1a4
HBASE-25929 RegionServer JVM crash when compaction ( #3318 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-03 17:20:08 +08:00
a4e8ee183e
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime ( #3302 )
...
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
2021-06-01 12:41:15 -07:00
d4285be5c1
HBASE-25932: Ensure replication reads the trailer bytes from WAL. ( #3332 ) ( #3345 )
...
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).
While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c7786
2021-06-01 08:04:12 -07:00
0ede56d24a
HBASE-25903 ReadOnlyZKClient APIs - CompletableFuture.get() calls can cause threads to hang forver when ZK client create throws Non IOException ( #3293 )
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-05-31 11:39:14 +05:30
920a9568b4
HBASE-25948 Remove deprecated ZK command 'rmr' in hbase-cleanup.sh ( #3343 )
...
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit bd4c4cec1f
2021-05-31 11:06:26 +05:30
9beb278d8b
HBASE-25941 TestRESTServerSSL fails because of jdk bug ( #3335 )
...
Signed-off-by: Wei-Chiu Chuang weichiu@apache.org
2021-05-30 21:33:34 -07:00
59513e81b7
HBASE-25942: Get rid of null regioninfo in wrapped connection exceptions ( #3337 )
...
Reviewed-by: Tamanna Mehta
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 528f543e23
2021-05-30 21:08:55 -07:00
1a45b743a0
HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO ( #3334 )
...
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-29 09:04:08 -07:00
5157521213
HBASE-25927: Fix the log messages by not stringifying the exceptions in log ( #3338 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 9a2027bf71
2021-05-28 21:34:14 -07:00
2a84fd40d3
HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one ( #3329 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-29 11:10:52 +08:00
fada9884a4
HBASE-25908 Exclude jakarta.activation-api ( #3299 )
...
Reapply of a revert because commit message was in wrong format
Signed-off-by: stack <stack@apache.org>
2021-05-28 13:27:13 -07:00
5ce6005550
Revert "Subject: [PATCH] HBASE-25908 Exclude jakarta.activation-api ( #3299 )"
...
This reverts commit 1e01c94ba3
2021-05-28 13:27:13 -07:00
bbe9558040
HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.count` to `hbase.normalizer.merge.min.region.count`
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-28 13:10:20 -07:00
5aa39a975c
HBASE-25910 - Fix port assignment test ( #3308 )
...
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-28 20:53:20 +05:30
a1cac20e95
HBASE-25928 TestHBaseConfiguration#testDeprecatedConfigurations is broken with Hadoop 3.3 ( #3320 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-05-28 07:27:39 -07:00
b4e8b91414
HBASE-25718 Backport "HBASE-25705 Convert proto to RSGroupInfo is costly" ( #3109 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-05-28 19:23:25 +08:00
1e01c94ba3
Subject: [PATCH] HBASE-25908 Exclude jakarta.activation-api ( #3299 )
...
Signed-off-by: stack <stack@apache.org>
2021-05-27 18:47:20 -07:00
a3aec54e06
Revert "HBASE-25908 Exclude jakarta.activation-api ( #3299 )"
...
This reverts commit 21e9c6ae9e
2021-05-27 18:40:52 -07:00
21e9c6ae9e
HBASE-25908 Exclude jakarta.activation-api ( #3299 )
...
Signed-off-by: stack <stack@apache.org>
2021-05-27 18:14:36 -07:00
cad9ae386a
HBASE-25933 Log trace raw exception, instead of cause message in NettyRpcServerRequestDecoder ( #3323 )
...
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-05-27 20:03:17 +01:00
3cf0678aee
HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code ( #3313 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-27 16:25:01 +08:00
e265eccf20
HBASE-25924 Re-compute size of WAL file while removing from WALEntryStream ( #3315 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-05-26 10:42:03 -07:00
f7358cd618
HBASE-25841 Add basic jshell support (addendum)
2021-05-25 10:10:11 -07:00
563ebc214a
HBASE-25534 Honor TableDescriptor settings earlier in normalization ( #2917 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-25 08:24:05 -07:00
2cb6cc8a6c
HBASE-25894 Improve the performance for region load and region count related cost functions ( #3276 )
...
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
2021-05-25 18:07:33 +08:00
8754e88fa7
HBASE-25906 UI of master-status to show recent history of balancer desicion ( #3303 ) ( #3296 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-25 12:39:47 +05:30
073f23e2bd
HBASE-25513 When the table is turned on normalize, the first region may not be merged even the size is 0 ( #2887 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-24 17:27:06 -07:00
1649013b99
HBASE-25898 RS getting aborted due to NPE in Replication WALEntryStream ( #3292 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
2021-05-24 23:44:32 +05:30
76d56bc42e
HBASE-25899 Improve efficiency of SnapshotHFileCleaner ( #3280 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-24 22:18:43 +08:00
1c1acee2fb
HBASE-25873 Refactor and cleanup the code for CostFunction ( #3274 )
...
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
2021-05-24 22:13:01 +08:00
16f865f65d
HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky ( #3160 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-22 22:20:47 +08:00
a19dee1eed
HBASE-25892: 'False' should be 'True' in auditlog of listLabels ( #3273 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-22 17:25:43 +08:00
375aee2327
HBASE-25817 Memory leak from thrift server hashMap ( #3257 )
...
Use GuavaCache in thrift server hashmap
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-22 10:51:09 +08:00
926c1132c0
HBASE-25848: Add flexibility to backup replication in case replication filter throws an exception
...
Signed-off-by: Xu Cang <xcang@salesforce.com>
2021-05-21 12:14:35 -07:00
2d591ab3c4
HBASE-25869 WAL value compression ( #3244 )
...
WAL storage can be expensive, especially if the cell values
represented in the edits are large, consisting of blobs or
significant lengths of text. Such WALs might need to be kept around
for a fairly long time to satisfy replication constraints on a space
limited (or space-contended) filesystem.
We have a custom dictionary compression scheme for cell metadata that
is engaged when WAL compression is enabled in site configuration.
This is fine for that application, where we can expect the universe
of values and their lengths in the custom dictionaries to be
constrained. For arbitrary cell values it is better to use one of the
available compression codecs, which are suitable for arbitrary albeit
compressible data.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-21 11:53:36 -07:00
7b374b7c76
HBASE-25827 Per Cell TTL tags get duplicated with increments causing tags length overflow ( #3290 )( #3210 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-21 22:58:32 +05:30
litao
binlijin
Toshihiro Suzuki
Duo Zhang
Andrew Purtell
Michael Stack
Mate Szalay-Beko
bsglz
Xiaolin Ha
Pankaj
Haoning Sun
meiyi
Bharath Vissapragada
Anoop Sam John
mokai
Sandeep Pal
stack
Baiqiang Zhao
Victor
Wei-Chiu Chuang
Wellington Ramos Chevreuil
Rushabh Shah
Nick Dimiduk
GeorryHuang
caoliqing
sambong0113
Sandeep Pal