Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,881 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

282 Commits

Author SHA1 Message Date
Josh Elser e8d62139d5 HBASE-26273 Force ReadType.STREAM when the user does not explicitly set a ReadType on the Scan for a Snapshot-based Job 
HBase 2 moved over Scans to use PREAD by default instead of STREAM like
HBase 1. In the context of a MapReduce job, we can generally expect that
clients using the InputFormat (batch job) would be reading most of the
data for a job. Cater to them, but still give users who want PREAD the
ability to do so.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
 2021-09-13 19:10:25 -04:00
bitterfox e139d31612
HBASE-26272 Fix TestTableMapReduceUntil failure in branch-2 (#3676) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-13 10:35:22 +08:00
Wellington Ramos Chevreuil 126f6bc520 HBASE-26276 Allow HashTable/SyncTable to perform rawScan when comparing cells (#3673) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry-picked from commit ee632bdcae)
 2021-09-10 16:46:27 +01:00
huaxiangsun 21d9741adb
HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat (#3661) (#3668) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-09-09 10:43:36 -07:00
Rushabh Shah 820ba21b2a HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
 2021-09-02 10:26:22 -07:00
Tak Lon (Stephen) Wu 625093f907 HBASE-26168 Backport HBASE-25811 "The client integration test is failing after HBASE-22120 merged" into branch-2 (#3634) 
move opentelemetry jars to client-facing-thirdparty
add opentelemetry jars when init map reduce job dependencies

original commit 8d2a0efb7a

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 665305cc3b HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529) 
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-01 15:29:09 -07:00
bitterfox 008ffd2130 HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster (#3592) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Rushabh Shah <shahrs87@gmail.com>
 2021-09-01 20:48:10 +08:00
bitterfox f4edf59833
HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress (#3625) 
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry-picked from commit d781113a08)
 2021-08-26 10:59:23 +01:00
bitterfox e08b3642a7 HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive (#3582) 
Signed-off-by: stack <stack@apache.org>
 2021-08-16 10:27:32 -07:00
Tak Lon (Stephen) Wu c11a3e1b39 Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)" 
This reverts commit f049301606.
 2021-08-04 15:55:13 -07:00
Tak Lon (Stephen) Wu f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529) 
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-07-29 09:15:10 -07:00
huaxiangsun 68549338d0
HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat (#3519) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2021-07-22 12:03:52 -07:00
Andrew Purtell a4e8ee183e HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302) 
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
	hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 2021-06-01 12:41:15 -07:00
Nick Dimiduk aab6e1d0ff
HBASE-25861 Correct the usage of Configuration#addDeprecation (#3249) 
Co-authored-by: Baiqiang Zhao <zbq.dean@gmail.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-18 06:53:31 -07:00
stack 04c30bda88 HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars (#3184) 
Need to add to allowed-licenses list too....

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-04-27 08:41:15 -07:00
bitterfox 150e02ee58
HBASE-25671 Backport HBASE-25608 to branch-2 (#3058) 
Signed-off-by: stack <stack@apache.org>
 2021-03-18 12:00:25 -07:00
Duo Zhang 4c8de44bf9 HBASE-25662 Fix spotbugs warning in RoundRobinTableInputFormat (#3050) 2021-03-15 08:16:18 -07:00
sudhir-reddy 40d7e024c7 HBASE-25566 RoundRobinTableInputFormat (#2947) 
Co-authored-by: stack <stack@apache.org>
Co-authored-by: sudhir-reddy <sudhir-reddy>
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-03-11 20:54:56 -08:00
Peter Somogyi c1cc8469dc HBASE-25611 ExportSnapshot chmod flag uses value as decimal (#3003) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-03-02 11:25:58 +01:00
Tak Lon (Stephen) Wu 2bb7beb448
Backport "HBASE-25249 Adding StoreContext" to branch-2 (#2869) 
Co-authored-by: Abhishek Khanna <akkhanna@amazon.com>

Signed-off-by: Zach York <zyork@apache.org>
 2021-01-13 20:13:43 -08:00
Viraj Jasani 0788547fea
HBASE-25474 : Bump HBase version on branch-2 (#2871) 
Signed-off-by: stack <stack@apache.org>
 2021-01-12 10:20:22 +05:30
shahrs87 9b4be47c24
HBASE-25246 Backup/Restore hbase cell tags (#2766) 
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-16 18:59:19 +05:30
Viraj Jasani a95c0c6537
Revert "HBASE-25246 Backup/Restore hbase cell tags" 
This reverts commit 34721c42ec.
 2020-12-03 21:17:11 +05:30
shahrs87 34721c42ec
HBASE-25246 Backup/Restore hbase cell tags 
Closes #2706

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-02 21:30:46 +05:30
clarax 4c8fa47b28
HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2655) 
* HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)


Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
 2020-11-30 22:51:57 -08:00
stack 01337e4104 Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) (#2682)" 
This reverts commit 5fcbf1f9f2.
 2020-11-30 16:45:59 -08:00
Michael Stack 5fcbf1f9f2 HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) (#2682) 
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-11-30 10:29:29 -08:00
Andrew Purtell 1b6399a480
HBASE-24640 [branch-2] Purge use of VisibleForTesting (#2696) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-11-25 14:15:32 -08:00
stack 4b9a14d9f3 Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)" 
This reverts commit aa324c2853.
 2020-11-14 11:15:29 -08:00
clarax aa324c2853 HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
 2020-11-14 10:46:34 -08:00
Sandeep Pal 729518af9d
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2610) 
Patch fixes the single table input format case.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-11-03 07:40:25 -08:00
Michael Stack 45e10d43fe HBASE-22976 [HBCK2] Add RecoveredEditsPlayer (#2504) 
Make it so WALPlayer can replay recovered.edits files.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
 Allow for WAL files that do NOT have a startime in their name.
 Use the 'generic' WAL-filename parser instead of the one that
 used be local here. Implement support for 'startTime' filter.
 Previous was just not implemented.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
 Checkstyle.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
 Use the new general WAL name timestamp parser.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility for parsing timestamp from WAL filename.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
 Export attributes about the local recovered.edits test file
 so other tests can play with it.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-10-09 08:54:53 -07:00
Michael Stack f52e5c797f
HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if it is doing anything (#2468) 
Add MR counters so operator can see if WALPlayer run actually did
anything. Fix bugs in usage (it enforced two args though usage
describes allowing one arg only). Clean up usage output. In
particular add note on wal file separator as hbase by default uses
the ',' in its WAL file names which could befuddle operator
trying to do simple import.

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
 2020-09-30 09:34:15 -07:00
Viraj Jasani ddddd2a822
HBASE-24831 : Addendum for source compatibility 
Closes #2403

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-09-15 20:20:26 +05:30
Yechao Chen e042cabfb3 HBASE-24831 Avoid invoke Counter using reflection in SnapshotInputFormat (#2209) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-09-02 16:25:04 +08:00
Duo Zhang 0be2b6ce55 HBASE-24945 Remove MetaTableAccessor.getRegionCount (#2303) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-08-25 21:33:15 +08:00
Guanghao Zhang 4667a971b1 HBASE-24912 Enlarge MemstoreFlusherChore/CompactionChecker period for unit test (#2285) 
Signed-off-by: stack <stack@apache.org>
 2020-08-21 12:42:11 +08:00
huaxiangsun bd30555ac8
HBASE-24824 Add more stats in PE for read replica (#2205) (#2212) 
Signed-off-by: stack <stack@apache.org>
 2020-08-10 09:58:50 -07:00
Viraj Jasani 4a2f3f89f4
HBASE-24788 : TableOutputFormat source compatibility (ADDENDUM) 
Brought back InterruptedException in throws clause of getRecordWriter and checkOutputSpecs of TableOutputFormat
 2020-08-08 14:30:40 +05:30
Sandeep Pal 6c61c50df7
HBASE-24788: Fix the connection leaks on getting hbase admin from unclosed connection (#2173) 
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-07-30 10:32:32 -07:00
Wellington Ramos Chevreuil 017c374519 HBASE-21773 - Addendum - Bring back "public static Job createSubmitta… (#1953) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

(cherry picked from commit edf75f2535)
 2020-06-23 15:17:22 +01:00
Wellington Ramos Chevreuil ef984cb18e HBASE-24524 SyncTable logging improvements (#1878) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit d8247ebae3)
 2020-06-10 09:41:33 +01:00
Sambit Mohapatra e5ec59ba2c HBASE-24340 PerformanceEvaluation options should not mandate any specific order 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2020-06-09 11:37:01 +05:30
stack 7182bb83ef HBASE-24493 [flakey test] TestExportSnapshot family of tests failing due to timeout in AbstractDelegationTokenSecretManager$ExpiredTokenRemover 
Disable TestExportSnapshot* tests because they timeout on occasion
staring yarn jobhistoryserver.
 2020-06-08 10:25:40 -07:00
niuyulin 46e035f6bd HBASE-24504 refactor call setupCluster/tearDownCluster in TestTableSnapshotInputFormat (#1851) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-05 15:53:39 +08:00
niuyulin 61eb7e5003 HBASE-24387 TableSnapshotInputFormatImpl support row limit on each InputSplit (#1731) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-05-25 10:05:09 +08:00
shahrs87 941cef71c2
HBASE-24190 : Make kerberos value of hbase.security.authentication property case insensitive (#1687) 
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-14 11:46:33 +05:30
Viraj Jasani b78f4367f7
Revert "Make kerberos value of hbase.security.authentication property case insensitive (#1687)" 
This reverts commit c03ec837e7.
 2020-05-14 11:45:23 +05:30
Duo Zhang dc2146069c
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for … (#1697) 
Signed-off-by: stack <stack@apache.org>
 2020-05-13 17:59:21 +08:00