Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,967 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

249 Commits

Author SHA1 Message Date
Andrew Purtell 210517040b HBASE-26582 Prune use of Random and SecureRandom objects (#4118) 
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.

Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.

Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-08 16:12:08 -08:00
Mike Drob c3ac78691e HBASE-26622 Update error-prone to 2.10 
Author:    Mike Drob <mdrob@apache.org>
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-03 11:52:16 +01:00
meiyi b68c4aae06 HBASE-26646 WALPlayer should obtain token from filesystem (#4003) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-07 18:00:58 +08:00
meiyi a7c4a394a9 HBASE-26625 ExportSnapshot tool failed to copy data files for tables with merge region (#3981) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-30 12:17:40 +08:00
Ruanhui 3e14c8d6e6 HBASE-26485 Introduce a method to clean restore directory after Snapshot Scan (#3877) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-27 20:42:01 +08:00
Istvan Toth 5fbb089c92 HBASE-26398 CellCounter fails for large tables filling up local disk (#3798) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 60254bc184)
 2021-10-28 09:50:13 +01:00
bitterfox a6329385a0 HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME (#3647) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-28 23:16:33 +08:00
Josh Elser 74eff79c27 HBASE-26273 Force ReadType.STREAM when the user does not explicitly set a ReadType on the Scan for a Snapshot-based Job 
HBase 2 moved over Scans to use PREAD by default instead of STREAM like
HBase 1. In the context of a MapReduce job, we can generally expect that
clients using the InputFormat (batch job) would be reading most of the
data for a job. Cater to them, but still give users who want PREAD the
ability to do so.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
 2021-09-13 19:31:48 -04:00
bitterfox ff0a33ec40 HBASE-26272 Fix TestTableMapReduceUntil failure in branch-2 (#3676) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-13 10:48:35 +08:00
Wellington Ramos Chevreuil 30ae596308 HBASE-26276 Allow HashTable/SyncTable to perform rawScan when comparing cells (#3673) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry-picked from commit ee632bdcae)
 2021-09-10 16:47:58 +01:00
huaxiangsun be23541f11
HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat (#3661) (#3669) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-09-09 10:43:53 -07:00
Rushabh Shah f9a9b6eaee HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
 2021-09-02 12:08:49 -07:00
bitterfox 875f3afc2a HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster (#3592) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Rushabh Shah <shahrs87@gmail.com>
 2021-09-01 20:48:26 +08:00
bitterfox 20279c704d HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress (#3625) 
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry-picked from commit d781113a08)
 2021-08-26 11:03:55 +01:00
bitterfox 6cc26c2487 HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive (#3582) 
Signed-off-by: stack <stack@apache.org>
 2021-08-16 10:30:15 -07:00
huaxiangsun d10847cc3f
HBASE-26108 add option to disable scanMetrics in TableSnapshotInputFormat (#3520) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2021-07-22 12:22:31 -07:00
bitterfox 63f1580015 HBASE-25671 Backport HBASE-25608 to branch-2 (#3058) 
Signed-off-by: stack <stack@apache.org>
 2021-03-18 12:01:06 -07:00
Duo Zhang 3fb24a0c2b HBASE-25662 Fix spotbugs warning in RoundRobinTableInputFormat (#3050) 2021-03-15 08:16:59 -07:00
sudhir-reddy f54dd287dc HBASE-25566 RoundRobinTableInputFormat (#2947) 
Co-authored-by: stack <stack@apache.org>
Co-authored-by: sudhir-reddy <sudhir-reddy>
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-03-11 20:55:56 -08:00
Peter Somogyi 0ffd42f5fd HBASE-25611 ExportSnapshot chmod flag uses value as decimal (#3003) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-03-02 11:33:39 +01:00
Tak Lon (Stephen) Wu 83a6ca54bd Backport "HBASE-25249 Adding StoreContext" to branch-2 (#2869) 
Co-authored-by: Abhishek Khanna <akkhanna@amazon.com>

Signed-off-by: Zach York <zyork@apache.org>
 2021-01-14 10:56:26 -08:00
shahrs87 47efdbb871
HBASE-25246 Backup/Restore hbase cell tags (#2766) 
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-17 12:55:41 +05:30
Viraj Jasani 75421af689
Revert "HBASE-25246 Backup/Restore hbase cell tags" 
This reverts commit 34721c42ec.
 2020-12-03 21:18:03 +05:30
shahrs87 34721c42ec
HBASE-25246 Backup/Restore hbase cell tags 
Closes #2706

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-02 21:30:46 +05:30
clarax 4c8fa47b28
HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2655) 
* HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)


Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
 2020-11-30 22:51:57 -08:00
stack 01337e4104 Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) (#2682)" 
This reverts commit 5fcbf1f9f2.
 2020-11-30 16:45:59 -08:00
Michael Stack 5fcbf1f9f2 HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) (#2682) 
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-11-30 10:29:29 -08:00
Andrew Purtell 1b6399a480
HBASE-24640 [branch-2] Purge use of VisibleForTesting (#2696) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-11-25 14:15:32 -08:00
stack 4b9a14d9f3 Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)" 
This reverts commit aa324c2853.
 2020-11-14 11:15:29 -08:00
clarax aa324c2853 HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
 2020-11-14 10:46:34 -08:00
Sandeep Pal 729518af9d
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2610) 
Patch fixes the single table input format case.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-11-03 07:40:25 -08:00
Michael Stack 45e10d43fe HBASE-22976 [HBCK2] Add RecoveredEditsPlayer (#2504) 
Make it so WALPlayer can replay recovered.edits files.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
 Allow for WAL files that do NOT have a startime in their name.
 Use the 'generic' WAL-filename parser instead of the one that
 used be local here. Implement support for 'startTime' filter.
 Previous was just not implemented.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
 Checkstyle.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
 Use the new general WAL name timestamp parser.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility for parsing timestamp from WAL filename.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
 Export attributes about the local recovered.edits test file
 so other tests can play with it.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-10-09 08:54:53 -07:00
Michael Stack f52e5c797f
HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if it is doing anything (#2468) 
Add MR counters so operator can see if WALPlayer run actually did
anything. Fix bugs in usage (it enforced two args though usage
describes allowing one arg only). Clean up usage output. In
particular add note on wal file separator as hbase by default uses
the ',' in its WAL file names which could befuddle operator
trying to do simple import.

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
 2020-09-30 09:34:15 -07:00
Viraj Jasani ddddd2a822
HBASE-24831 : Addendum for source compatibility 
Closes #2403

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-09-15 20:20:26 +05:30
Yechao Chen e042cabfb3 HBASE-24831 Avoid invoke Counter using reflection in SnapshotInputFormat (#2209) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-09-02 16:25:04 +08:00
Duo Zhang 0be2b6ce55 HBASE-24945 Remove MetaTableAccessor.getRegionCount (#2303) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-08-25 21:33:15 +08:00
Guanghao Zhang 4667a971b1 HBASE-24912 Enlarge MemstoreFlusherChore/CompactionChecker period for unit test (#2285) 
Signed-off-by: stack <stack@apache.org>
 2020-08-21 12:42:11 +08:00
huaxiangsun bd30555ac8
HBASE-24824 Add more stats in PE for read replica (#2205) (#2212) 
Signed-off-by: stack <stack@apache.org>
 2020-08-10 09:58:50 -07:00
Viraj Jasani 4a2f3f89f4
HBASE-24788 : TableOutputFormat source compatibility (ADDENDUM) 
Brought back InterruptedException in throws clause of getRecordWriter and checkOutputSpecs of TableOutputFormat
 2020-08-08 14:30:40 +05:30
Sandeep Pal 6c61c50df7
HBASE-24788: Fix the connection leaks on getting hbase admin from unclosed connection (#2173) 
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-07-30 10:32:32 -07:00
Wellington Ramos Chevreuil 017c374519 HBASE-21773 - Addendum - Bring back "public static Job createSubmitta… (#1953) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

(cherry picked from commit edf75f2535)
 2020-06-23 15:17:22 +01:00
Wellington Ramos Chevreuil ef984cb18e HBASE-24524 SyncTable logging improvements (#1878) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit d8247ebae3)
 2020-06-10 09:41:33 +01:00
Sambit Mohapatra e5ec59ba2c HBASE-24340 PerformanceEvaluation options should not mandate any specific order 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2020-06-09 11:37:01 +05:30
stack 7182bb83ef HBASE-24493 [flakey test] TestExportSnapshot family of tests failing due to timeout in AbstractDelegationTokenSecretManager$ExpiredTokenRemover 
Disable TestExportSnapshot* tests because they timeout on occasion
staring yarn jobhistoryserver.
 2020-06-08 10:25:40 -07:00
niuyulin 46e035f6bd HBASE-24504 refactor call setupCluster/tearDownCluster in TestTableSnapshotInputFormat (#1851) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-05 15:53:39 +08:00
niuyulin 61eb7e5003 HBASE-24387 TableSnapshotInputFormatImpl support row limit on each InputSplit (#1731) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-05-25 10:05:09 +08:00
shahrs87 941cef71c2
HBASE-24190 : Make kerberos value of hbase.security.authentication property case insensitive (#1687) 
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-14 11:46:33 +05:30
Viraj Jasani b78f4367f7
Revert "Make kerberos value of hbase.security.authentication property case insensitive (#1687)" 
This reverts commit c03ec837e7.
 2020-05-14 11:45:23 +05:30
Duo Zhang dc2146069c
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for … (#1697) 
Signed-off-by: stack <stack@apache.org>
 2020-05-13 17:59:21 +08:00
shahrs87 c03ec837e7
Make kerberos value of hbase.security.authentication property case insensitive (#1687) 
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-11 11:10:29 +05:30