99f2ab5aa8
HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 10:00:31 +08:00
e047ef8c68
HBASE-23330: Fix delegation token fetch with MasterRegistry ( #1084 ) ( #4598 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit d8b3f55518
2022-07-06 08:10:50 +08:00
d28f06a560
HBASE-27143 Add hbase-unsafe as a dependency for a MR job triggered by hbase shell ( #4554 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-21 13:39:20 -07:00
bdaa4486f3
HBASE-26956 ExportSnapshot tool supports removing TTL ( #4538 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-21 09:19:08 +08:00
57d31224f6
HBASE-26923 PerformanceEvaluation support encryption option ( #4489 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b7065c1cf5
2022-06-11 21:14:40 +08:00
5cfce692a5
HBASE-27003 Optimize log format for PerformanceEvaluation ( #4411 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-05-10 22:32:36 -07:00
1a5b1b266c
HBASE-26899 Run spotless:apply
2022-05-01 22:41:49 +08:00
e7eb628025
HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless ( #4324 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org
(cherry picked from commit 3ae0d9012c
2022-04-09 23:13:49 +08:00
03b5529feb
HBASE-26916 Fix missing braces warnings in DefaultVisibilityExpressionResolver ( #4313 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit c4ff355915
2022-04-03 23:04:27 +08:00
ef40c044d1
HBASE-26878 TableInputFormatBase should cache RegionSizeCalculator ( #4271 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-24 16:08:33 -07:00
fdc7c49c01
HBASE-26720 ExportSnapshot should validate the source snapshot before copying files ( #4216 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-16 20:29:48 +08:00
340cc6c6f1
HBASE-26802 Backport the log4j2 changes to branch-2 ( #4166 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-11 11:17:43 -08:00
300f9b9576
HBASE-26582 Prune use of Random and SecureRandom objects ( #4118 )
...
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.
Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.
Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-08 15:22:00 -08:00
4644efb9f3
HBASE-26691 Replacing log4j with reload4j for branch-2.x ( #4050 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-04 12:06:34 -08:00
8a9e083994
HBASE-26622 Update error-prone to 2.10
...
Author: Mike Drob <mdrob@apache.org>
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-03 11:51:51 +01:00
a8b68c9b88
HBASE-26690 Modify FSTableDescriptors to not rely on renaming when writing TableDescriptor ( #4054 )
...
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-01-27 13:38:11 +08:00
d25fa46c8f
HBASE-26646 WALPlayer should obtain token from filesystem ( #4003 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-07 17:44:41 +08:00
5b08db92c7
HBASE-26625 ExportSnapshot tool failed to copy data files for tables with merge region ( #3981 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-30 11:08:36 +08:00
c6a287930b
HBASE-26564 Retire the method visitLogEntryBeforeWrite without RegionInfo in WALActionListner ( #3985 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-28 21:42:00 +08:00
b1bc5f3a5c
Renumber to 2.6.0-SNAPSHOT after branching branch-2.5
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-08 16:54:32 -08:00
4fb3e304e2
HBASE-26485 Introduce a method to clean restore directory after Snapshot Scan ( #3877 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-27 20:41:49 +08:00
4fb8a7f7e5
HBASE-26446 CellCounter should report serialized cell size counts too ( #3841 )
...
Add the following stats for a given table:
- 7. Total size of serialized cells of each CF.
- 8. Total size of serialized cells of each qualifier.
- 9. Total size of serialized cells across all rows.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-11-15 16:46:01 -08:00
7353c281fc
HBASE-26398 CellCounter fails for large tables filling up local disk ( #3798 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-10-28 09:48:28 +01:00
b6bb18022f
HBASE-26316 Per-table or per-CF compression codec setting overrides ( #3730 )
...
We get and retain Compressor instances in HFileBlockDefaultEncodingContext,
and could in theory call Compressor#reinit when setting up the context,
to update compression parameters like level and buffer size, but we do
not plumb through the CompoundConfiguration from the Store into the
encoding context. As a consequence we can only update codec parameters
globally in system site conf files.
Fine grained configurability is important for algorithms like ZStandard
(ZSTD), which offers more than 20 compression levels, where at level 1
it is almost as fast as LZ4, and where at higher levels it utilizes
computationally expensive techniques to rival LZMA at compression ratio
but trades off significantly for reduced compresson throughput. The ZSTD
level that should be set for a given column family or table will vary by
use case.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdDecompressor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/compress/HFileTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekToBlockWithEncoders.java
2021-10-19 13:59:04 -07:00
18b9fa8a3c
HBASE-26259 Fallback support to pure Java compression ( #3691 )
...
This change introduces provided compression codecs to HBase as
new Maven modules. Each module provides compression codec support
that formerly required Hadoop native codecs, which in turn relies
on native code integration, which may or may not be available on
a given hardware platform or in an operational environment. We
now provide codecs in the HBase distribution for users whom for
whatever reason cannot or do not wish to deploy the Hadoop native
codecs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestCompressedWAL.java
2021-10-06 13:48:33 -07:00
786e09a936
HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME ( #3647 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-28 23:16:20 +08:00
e8d62139d5
HBASE-26273 Force ReadType.STREAM when the user does not explicitly set a ReadType on the Scan for a Snapshot-based Job
...
HBase 2 moved over Scans to use PREAD by default instead of STREAM like
HBase 1. In the context of a MapReduce job, we can generally expect that
clients using the InputFormat (batch job) would be reading most of the
data for a job. Cater to them, but still give users who want PREAD the
ability to do so.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
2021-09-13 19:10:25 -04:00
e139d31612
HBASE-26272 Fix TestTableMapReduceUntil failure in branch-2 ( #3676 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-13 10:35:22 +08:00
126f6bc520
HBASE-26276 Allow HashTable/SyncTable to perform rawScan when comparing cells ( #3673 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry-picked from commit ee632bdcae
2021-09-10 16:46:27 +01:00
21d9741adb
HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat ( #3661 ) ( #3668 )
...
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-09-09 10:43:36 -07:00
820ba21b2a
HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. ( #3636 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
2021-09-02 10:26:22 -07:00
625093f907
HBASE-26168 Backport HBASE-25811 "The client integration test is failing after HBASE-22120 merged" into branch-2 ( #3634 )
...
move opentelemetry jars to client-facing-thirdparty
add opentelemetry jars when init map reduce job dependencies
original commit 8d2a0efb7a
2021-09-01 15:29:09 -07:00
665305cc3b
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
008ffd2130
HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster ( #3592 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Rushabh Shah <shahrs87@gmail.com>
2021-09-01 20:48:10 +08:00
f4edf59833
HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress ( #3625 )
...
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry-picked from commit d781113a08
2021-08-26 10:59:23 +01:00
e08b3642a7
HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive ( #3582 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-16 10:27:32 -07:00
c11a3e1b39
Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )"
...
This reverts commit f049301606
2021-08-04 15:55:13 -07:00
f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-07-29 09:15:10 -07:00
68549338d0
HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat ( #3519 )
...
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2021-07-22 12:03:52 -07:00
a4e8ee183e
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime ( #3302 )
...
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
2021-06-01 12:41:15 -07:00
aab6e1d0ff
HBASE-25861 Correct the usage of Configuration#addDeprecation ( #3249 )
...
Co-authored-by: Baiqiang Zhao <zbq.dean@gmail.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-18 06:53:31 -07:00
04c30bda88
HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars ( #3184 )
...
Need to add to allowed-licenses list too....
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-04-27 08:41:15 -07:00
150e02ee58
HBASE-25671 Backport HBASE-25608 to branch-2 ( #3058 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:00:25 -07:00
4c8de44bf9
HBASE-25662 Fix spotbugs warning in RoundRobinTableInputFormat ( #3050 )
2021-03-15 08:16:18 -07:00
40d7e024c7
HBASE-25566 RoundRobinTableInputFormat ( #2947 )
...
Co-authored-by: stack <stack@apache.org>
Co-authored-by: sudhir-reddy <sudhir-reddy>
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-03-11 20:54:56 -08:00
c1cc8469dc
HBASE-25611 ExportSnapshot chmod flag uses value as decimal ( #3003 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-02 11:25:58 +01:00
2bb7beb448
Backport "HBASE-25249 Adding StoreContext" to branch-2 ( #2869 )
...
Co-authored-by: Abhishek Khanna <akkhanna@amazon.com>
Signed-off-by: Zach York <zyork@apache.org>
2021-01-13 20:13:43 -08:00
0788547fea
HBASE-25474 : Bump HBase version on branch-2 ( #2871 )
...
Signed-off-by: stack <stack@apache.org>
2021-01-12 10:20:22 +05:30
9b4be47c24
HBASE-25246 Backup/Restore hbase cell tags ( #2766 )
...
Closes #2745
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-16 18:59:19 +05:30
a95c0c6537
Revert "HBASE-25246 Backup/Restore hbase cell tags"
...
This reverts commit 34721c42ec
2020-12-03 21:17:11 +05:30
Duo Zhang
Tak Lon (Stephen) Wu
XinSun
wenwj0
litao
Bryan Beaudreault
xicm
Andrew Purtell
Mike Drob
meiyi
Yutong Xiao
Ruanhui
Istvan Toth
bitterfox
Josh Elser
Wellington Ramos Chevreuil
huaxiangsun
Rushabh Shah
Nick Dimiduk
stack
bitterfox
sudhir-reddy
Peter Somogyi
Viraj Jasani