70 Commits

Author SHA1 Message Date
Josh Elser
15c398f7d2 HBASE-20223 Update to hbase-thirdparty 2.1.0
Remove commons-cli and commons-collections4 use. Account
for the newer internal protobuf version of 3.5.1.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-26 22:05:19 -04:00
Vladimir Rodionov
64ef120808 HBASE-19258 IntegrationTest for Backup and Restore
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-26 15:46:30 -07:00
tedyu
b30ff8196a HBASE-20224 Web UI is broken in standalone mode - addendum for hbase-backup module 2018-03-26 12:23:47 -07:00
Vladimir Rodionov
91075276e7 HBASE-19441: Implement retry logic around starting exclusive backup operation
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-21 09:34:45 -07:00
Chia-Ping Tsai
4f2133ee32 HBASE-20119 Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-16 01:21:38 +08:00
Mike Drob
f63a7ff635 HBASE-20180 Avoid Class::newInstance 2018-03-14 13:15:05 -05:00
Apekshit Sharma
84ee32c723 HBASE-20185 Fix ACL check for MasterRpcServices#execProcedure 2018-03-14 19:02:19 +05:30
BELUGA BEHR
000a84cb4a HBASE-20181 Logging and minor logic improvements in BackupLogCleaner
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-03-14 12:30:04 +05:30
Apekshit Sharma
810567f65a Revert "HBASE-20181 Logging and minor logic improvements in BackupLogCleaner" - missed author
This reverts commit 5bfc5745ab8e94d09851980dce2e243620f412ff.
2018-03-14 12:29:24 +05:30
Apekshit Sharma
5bfc5745ab HBASE-20181 Logging and minor logic improvements in BackupLogCleaner 2018-03-14 09:56:46 +05:30
Vladimir Rodionov
d5aaeee88b HBASE-19969: Improve fault tolerance in backup Merge operation
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-10 09:37:55 -08:00
tedyu
f89a1f7d7a HBASE-20121 Fix findbugs warning for RestoreTablesClient 2018-03-05 19:36:19 -08:00
Vladimir Rodionov
6cfa208add HBASE-17825: Backup further optimizations
Signed-off-by: Josh Elser <elserj@apache.org>
2018-02-28 09:14:15 -05:00
zhangduo
918599ef12 HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs 2018-01-29 08:43:56 +08:00
Jan Hentschel
c2236b77cb HBASE-19765 Fixed Checkstyle errors in hbase-backup 2018-01-28 13:45:45 +01:00
Vladimir Rodionov
a5601c8eac HBASE-19568: Restore of HBase table using incremental backup doesn't restore rows from an earlier incremental backup
Signed-off-by: Josh Elser <elserj@apache.org>
2018-01-12 13:13:17 -05:00
tedyu
cafd4e4ad7 HBASE-19478 Utilize multi-get to speed up WAL file checking in BackupLogCleaner (Toshihiro Suzuki) 2018-01-01 06:55:11 -08:00
Mike Drob
c3b4f788b1 HBASE-19552 find-and-replace thirdparty offset 2017-12-28 11:52:32 -06:00
Chia-Ping Tsai
01b1f48ccd HBASE-19644 add the checkstyle rule to reject the illegal imports 2017-12-28 04:10:42 +08:00
Chia-Ping Tsai
6b39062e86 HBASE-19628 ByteBufferCell should extend ExtendedCell 2017-12-27 22:26:40 +08:00
Balazs Meszaros
f572c4b80e HBASE-10092 Move up on to log4j2
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test

This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.

Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:21:33 -08:00
Jan Hentschel
d50ae03716 HBASE-19537 Removed unnecessary semicolons from hbase-backup 2017-12-19 21:03:07 +01:00
BELUGA BEHR
536187446d HBASE-19472 Remove ArrayUtil Class
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-13 19:02:07 +08:00
Peter Somogyi
7e5f3a516c HBASE-19213 Align check and mutate operations in Table and AsyncTable
- Deprecates old checkAnd*() operations in Table
- Adds Table#CheckAndMutateBuilder and implements it in HTable

Commiter note: When committing the patch, noticed redundant {@inheritDoc} being added in HTable.
Removed new and olds ones.
2017-12-12 20:37:24 -08:00
tedyu
82e278be55 HBASE-19469 Review Of BackupSystemTable (BELUGA BEHR) 2017-12-09 09:38:00 -08:00
tedyu
27ed4d8add HBASE-19417 Remove boolean return value from postBulkLoadHFile hook 2017-12-06 07:06:28 -08:00
tedyu
4c567b3c12 HBASE-19159 Backup should check permission for snapshot copy in advance (Janos Gub) 2017-12-04 11:59:18 -08:00
Michael Stack
5b13b624bb
HBASE-19181 LogRollBackupSubprocedure will fail if we use AsyncFSWAL instead of FSHLog (Vladimir Rodionov) 2017-11-16 15:04:07 -08:00
tedyu
cffbe4bef7 HBASE-19211 B&R: update configuration string in BackupRestoreConstants (Vladimir Rodionov) 2017-11-08 14:16:13 -08:00
tedyu
a5a38b7916 HBASE-19211 B&R: update configuration string in BackupRestoreConstants (Vladimir Rodionov) 2017-11-08 14:12:43 -08:00
tedyu
6712f8f632 HBASE-18870 Hbase Backup should set the details to MR job name (Vishal Khandelwal) 2017-10-30 10:35:52 -07:00
Apekshit Sharma
dd70cc3081 HBASE-19073 Cleanup CoordinatedStateManager
- Merged BaseCSM class into CSM interface
- Removed config hbase.coordinated.state.manager.class
- Since state manager is not pluggable anymore, we don't need start/stop/initialize to setup unknown classes. Our internal ZkCSM now requires Server in constructor itself. Makes the dependency clearer too.
- Removed CSM from HRegionServer and HMaster constructor. Although it's a step back from dependency injection, but it's more consistent with our current (not good)  pattern where we initialize everything in the ctor itself.

Change-Id: Ifca06bb354adec5b11ea1bad4707e014410491fc
2017-10-24 19:56:04 -07:00
Ramkrishna
70f4c5da47 HBSE-18945 Make a IA.LimitedPrivate interface for CellComparator (Ram) 2017-10-17 23:17:07 +05:30
tedyu
a6c9d371df HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov ) 2017-10-13 15:04:53 -07:00
tedyu
c4ced0b3d5 HBASE-18975 Fix backup / restore hadoop3 incompatibility (Vladimir Rodionov) 2017-10-11 12:26:34 -07:00
Amit
1ec6ece918 HBASE-18941 Confusing logging error around rerun of restore on an existing table.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-04 11:04:18 -07:00
Amit
2f2513255d HBASE-18932 Backup masking exception in a scenario and though it fails , it shows success message
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-04 08:18:08 -07:00
Ramkrishna
0a24178d06 HBASE-18649 Deprecate KV Usage in MR to move to Cells in 3.0 (ram) 2017-10-04 16:00:28 +05:30
tedyu
56830c3094 HBASE-18928 Backup delete command shows wrong number of deletes requested (Amit Kabra) 2017-10-03 14:07:58 -07:00
tedyu
1ff90d3b4b HBASE-18929 Hbase backup command doesn’t show debug option to enable backup in debug mode (Amit Kabra) 2017-10-03 10:11:27 -07:00
tedyu
ca2959824d HBASE-18887 After full backup passed on hdfs root and incremental failed, full backup cannot be cleaned (Vladimir Rodionov) 2017-09-28 10:20:02 -07:00
Chia-Ping Tsai
a11a35a113 HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:29 +08:00
Apekshit Sharma
2bf5bbd0a2 Revert HBASE-18843. Needs more work. Also, breaks building with Hadoop3. "Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov)"
This reverts commit 417dad646bf91922fe0117c83ea69aa501f3c2fc.
2017-09-27 20:39:15 -07:00
Apekshit Sharma
97513466c0 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:40:25 -07:00
tedyu
417dad646b HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov) 2017-09-26 20:58:16 -07:00
tedyu
976ef508f7 HBASE-18843 Add DistCp support to incremental backup with bulk loading - revert due to missing credit 2017-09-26 20:57:39 -07:00
tedyu
5df2fe7063 HBASE-18843 Add DistCp support to incremental backup with bulk loading 2017-09-26 20:55:19 -07:00
anoopsamjohn
3c7ab81072 HBASE-16769 Deprecate/remove PB references from MasterObserver and RegionServerObserver 2017-09-26 08:52:23 +05:30
tedyu
0ebd4b8433 HBASE-18876 Backup create command fails to take queue parameter as option (Amit Kabra) 2017-09-25 12:57:54 -07:00
zhangduo
a5f84430a3 HBASE-18825 Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface 2017-09-25 09:35:39 +08:00