Commit Graph

292 Commits

Author SHA1 Message Date
Peter Somogyi b24bd4095a
HBASE-25611 ExportSnapshot chmod flag uses value as decimal (#3003)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-02 11:23:10 +01:00
Tak Lon (Stephen) Wu 686b72c44e
HBASE-25249 Adding StoreContext (#2800)
Adding StoreContext which contains the metadata about the HStore. This
metadata can be used across the HFileWriter/Readers and other HStore
consumers without the need of passing around the complete store and
exposing its internals.

Co-authored-by: Abhishek Khanna <akkhanna@amazon.com>

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2021-01-08 20:16:45 -08:00
shahrs87 74d68180e6
HBASE-25246 Backup/Restore hbase cell tags
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-16 18:52:36 +05:30
Viraj Jasani f879d45fb0
Revert "HBASE-25246 Backup/Restore hbase cell tags"
This reverts commit 56eb5c9fc8.
2020-12-03 21:16:24 +05:30
shahrs87 56eb5c9fc8
HBASE-25246 Backup/Restore hbase cell tags
Closes #2706

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-02 20:50:30 +05:30
Michael Stack dfe3672266
HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) (#2682)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-30 10:28:34 -08:00
Andrew Purtell 1726160839
HBASE-24640 Purge use of VisibleForTesting (#2695)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-11-25 14:10:12 -08:00
stack 1528aacacd Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)"
This reverts commit 4ee2270452.
because on commit, I added 'Signed-off-by: Duo Zhang' when he nad
NOT signed off. My mistake.
2020-11-19 19:42:48 -08:00
clarax 4ee2270452
HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-11-14 10:45:42 -08:00
Sandeep Pal 5abbda1969
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2609)
Patch fixes the single table input format case.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-11-03 07:39:23 -08:00
stack c28220522d HBASE-25168 Unify WAL name timestamp parsers
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-10-12 07:52:26 -07:00
Michael Stack 665a8767a0
HBASE-22976 [HBCK2] Add RecoveredEditsPlayer (#2504)
Make it so WALPlayer can replay recovered.edits files.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
 Allow for WAL files that do NOT have a startime in their name.
 Use the 'generic' WAL-filename parser instead of the one that
 used be local here. Implement support for 'startTime' filter.
 Previous was just not implemented.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
 Checkstyle.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
 Use the new general WAL name timestamp parser.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility for parsing timestamp from WAL filename.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
 Export attributes about the local recovered.edits test file
 so other tests can play with it.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-10-09 08:46:05 -07:00
Michael Stack 01876071b9 HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if it is doing anything (#2468)
Add MR counters so operator can see if WALPlayer run actually did
anything. Fix bugs in usage (it enforced two args though usage
describes allowing one arg only). Clean up usage output. In
particular add note on wal file separator as hbase by default uses
the ',' in its WAL file names which could befuddle operator
trying to do simple import.

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-09-30 09:42:13 -07:00
Duo Zhang e386ec566f
HBASE-25054 Do not add non shaded protobuf as dependency in TableMapReduceUtil for master (#2412)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-17 14:55:36 +08:00
Viraj Jasani 325317ff9e
HBASE-24831 : Addendum for source compatibility
Closes #2403

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-09-15 20:18:40 +05:30
Yechao Chen 04de594138
HBASE-24831 Avoid invoke Counter using reflection in SnapshotInputFormat (#2209)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-02 16:22:44 +08:00
Duo Zhang 227084c41f
HBASE-24945 Remove MetaTableAccessor.getRegionCount (#2303)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-25 21:15:22 +08:00
Guanghao Zhang 54fe81eb56
HBASE-24912 Enlarge MemstoreFlusherChore/CompactionChecker period for unit test (#2285)
Signed-off-by: stack <stack@apache.org>
2020-08-21 12:41:20 +08:00
Viraj Jasani 3edfa1025c
HBASE-24788 : TableOutputFormat source compatibility (ADDENDUM)
Brought back InterruptedException in throws clause of getRecordWriter and checkOutputSpecs of TableOutputFormat
2020-08-08 14:21:17 +05:30
huaxiangsun 6ef90aad7d
HBASE-24824 Add more stats in PE for read replica (#2205)
Signed-off-by: stack <stack@apache.org>
2020-08-07 09:55:32 -07:00
Duo Zhang d2f5a5f27b
HBAE-24507 Remove HTableDescriptor and HColumnDescriptor (#2186)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2020-08-04 10:31:42 +08:00
YeChao Chen 8e33bb04bc
HBASE-24791 Improve HFileOutputFormat2 to avoid always call getTableRelativePath method (#2167)
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2020-08-03 13:32:48 +05:30
Sandeep Pal d65fb87a2e
HBASE-24788: Fix the connection leaks on getting hbase admin from unclosed connection (#2162)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-29 09:09:07 -07:00
Wellington Ramos Chevreuil b556343292 HBASE-21773 - Addendum - Bring back "public static Job createSubmitta… (#1953)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

(cherry picked from commit edf75f2535)
2020-06-23 16:14:59 +01:00
Wellington Ramos Chevreuil d8247ebae3
HBASE-24524 SyncTable logging improvements (#1878)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-10 09:36:30 +01:00
Sambit Mohapatra c718fa2365
HBASE-24340 PerformanceEvaluation options should not mandate any specific order
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-06-09 11:32:39 +05:30
stack 05ed4cd133 HBASE-24493 [flakey test] TestExportSnapshot family of tests failing due to timeout in AbstractDelegationTokenSecretManager$ExpiredTokenRemover
Disable TestExportSnapshot* tests because they timeout on occasion
staring yarn jobhistoryserver.
2020-06-08 14:20:12 -07:00
Duo Zhang c91829bb41
HBASE-24491 Remove HRegionInfo (#1830)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-06-05 22:19:01 +08:00
niuyulin 9cc3b79d5f
HBASE-24504 refactor call setupCluster/tearDownCluster in TestTableSnapshotInputFormat (#1851)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-05 14:52:29 +08:00
niuyulin 1be583f021
HBASE-24387 TableSnapshotInputFormatImpl support row limit on each InputSplit (#1731)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-05-25 09:55:48 +08:00
Duo Zhang 8601416ee8
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for modules other than hbase-assembly (#1640)
Signed-off-by: stack <stack@apache.org>
2020-05-12 12:03:30 +08:00
shahrs87 ff85daf1e2
[HBASE-24190] Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 10:54:45 +05:30
stack 2774510e2b HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625)
Addendum to fix failing MR tests; e.g. TestExportSnapshot.
2020-05-08 13:08:57 -07:00
Duo Zhang a9a1b9524d
HBASE-24304 Separate a hbase-asyncfs module (#1628)
Signed-off-by: stack <stack@apache.org>
2020-05-06 14:40:21 +08:00
Michael Stack 5488124be0 HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:36:01 -07:00
Wellington Ramos Chevreuil 3d59e328c8
HBASE-24302 Add an "ignoreTimestamps" option (defaulted to false) to … (#1623)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-05-04 11:24:43 +01:00
Duo Zhang 9f52e6b725
HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586)
Signed-off-by: stack <stack@apache.org>
2020-04-29 10:44:34 +08:00
Duo Zhang 6928674eb8
HBASE-24228 Merge the code in hbase-hadoop2-compat module to hbase-hadoop-compat (#1563)
Signed-off-by: stack <stack@apache.org>
2020-04-29 10:34:53 +08:00
Jan Hentschel 75c717d4c2
HBASE-23848 Removed deprecated setStopRow from Scan (#1184)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-22 15:15:17 +08:00
stack e7b5c34d28 HBASE-24226 Address other hard references to '/tmp' found in Configuration 2020-04-21 17:40:35 -07:00
stack 2d2e1d965d HBASE-24220 Allow that zk NOTEMPTY multi exception is retryable by running in-series
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java
 Cleanup checkstyle warnings. Don't depend on hbase-client
 ScannerCallable.

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
 Cut down on cluster resource usage.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/AbstractTestDLS.java
 Debug

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController3.java
 Debug

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
 Allow that NONEMPTY is retryable by running in series.
2020-04-20 15:51:11 -07:00
Duo Zhang 1f66806c96
HBASE-24170 Remove hadoop-2.0 profile (#1495)
Signed-off-by: stack <stack@apache.org>
2020-04-16 18:57:40 +08:00
stack 411c083f22 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum #2: The new test fails on origin.hbase.dir written by HTU up on
jenkins... has a 'tmp' in it. Fix.
2020-04-15 22:44:31 -07:00
stack 853a72bc46 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum: add test to check for '/tmp' references.
2020-04-15 10:55:40 -07:00
Viraj Jasani 2907260204
HBASE-24131 : TestExportSnapshot.testExportFileSystemState() changes with correct arg (ADDENDUM) 2020-04-09 10:32:55 +05:30
Michael Stack 082ebdd53c HBASE-24131 [Flakey Tests] TestExportSnapshot takes too long; up against 13min max (#1452)
Split TestExportSnapshot and TestVerifyReplication to two smaller tests
rather than one big one that can take 13minutes+ when contention.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-04-08 11:04:00 -07:00
Peter Somogyi 1784938af7
HBASE-24111 Enable CompactionTool executions on non-HDFS filesystems (#1427)
Signed-off-by: Josh Elser <elserj@apache.org>
2020-04-06 11:54:19 +02:00
Jan Hentschel 4ddf55d20a
HBASE-23843 Removed deprecated Scan(byte[]) from Scan
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-02 11:44:34 +02:00
Jan Hentschel fb17ecdd07
HBASE-23842 Removed deprecated Scan(byte[], byte[]) from Scan
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-01 13:11:21 +02:00
Jan Hentschel fded2b9ddc
HBASE-23846 Removed deprecated setMaxVersions(int) from Scan
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-31 10:18:27 +02:00