Commit Graph

79 Commits

Author SHA1 Message Date
Josh Elser 15c398f7d2 HBASE-20223 Update to hbase-thirdparty 2.1.0
Remove commons-cli and commons-collections4 use. Account
for the newer internal protobuf version of 3.5.1.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-26 22:05:19 -04:00
Vladimir Rodionov 64ef120808 HBASE-19258 IntegrationTest for Backup and Restore
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-26 15:46:30 -07:00
tedyu b30ff8196a HBASE-20224 Web UI is broken in standalone mode - addendum for hbase-backup module 2018-03-26 12:23:47 -07:00
Chia-Ping Tsai a6eeb26cc0 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:10:23 +08:00
Vladimir Rodionov 91075276e7 HBASE-19441: Implement retry logic around starting exclusive backup operation
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-21 09:34:45 -07:00
Chia-Ping Tsai 4f2133ee32 HBASE-20119 Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-16 01:21:38 +08:00
Mike Drob f63a7ff635 HBASE-20180 Avoid Class::newInstance 2018-03-14 13:15:05 -05:00
Apekshit Sharma 84ee32c723 HBASE-20185 Fix ACL check for MasterRpcServices#execProcedure 2018-03-14 19:02:19 +05:30
BELUGA BEHR 000a84cb4a HBASE-20181 Logging and minor logic improvements in BackupLogCleaner
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-03-14 12:30:04 +05:30
Apekshit Sharma 810567f65a Revert "HBASE-20181 Logging and minor logic improvements in BackupLogCleaner" - missed author
This reverts commit 5bfc5745ab.
2018-03-14 12:29:24 +05:30
Apekshit Sharma 5bfc5745ab HBASE-20181 Logging and minor logic improvements in BackupLogCleaner 2018-03-14 09:56:46 +05:30
Vladimir Rodionov d5aaeee88b HBASE-19969: Improve fault tolerance in backup Merge operation
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-10 09:37:55 -08:00
tedyu f89a1f7d7a HBASE-20121 Fix findbugs warning for RestoreTablesClient 2018-03-05 19:36:19 -08:00
Sean Busbey 2a65066b35 HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2018-03-02 09:25:10 -06:00
Vladimir Rodionov 6cfa208add HBASE-17825: Backup further optimizations
Signed-off-by: Josh Elser <elserj@apache.org>
2018-02-28 09:14:15 -05:00
zhangduo 918599ef12 HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs 2018-01-29 08:43:56 +08:00
Jan Hentschel c2236b77cb HBASE-19765 Fixed Checkstyle errors in hbase-backup 2018-01-28 13:45:45 +01:00
Vladimir Rodionov a5601c8eac HBASE-19568: Restore of HBase table using incremental backup doesn't restore rows from an earlier incremental backup
Signed-off-by: Josh Elser <elserj@apache.org>
2018-01-12 13:13:17 -05:00
tedyu cafd4e4ad7 HBASE-19478 Utilize multi-get to speed up WAL file checking in BackupLogCleaner (Toshihiro Suzuki) 2018-01-01 06:55:11 -08:00
Mike Drob c3b4f788b1 HBASE-19552 find-and-replace thirdparty offset 2017-12-28 11:52:32 -06:00
Chia-Ping Tsai 01b1f48ccd HBASE-19644 add the checkstyle rule to reject the illegal imports 2017-12-28 04:10:42 +08:00
Chia-Ping Tsai 6b39062e86 HBASE-19628 ByteBufferCell should extend ExtendedCell 2017-12-27 22:26:40 +08:00
Balazs Meszaros f572c4b80e HBASE-10092 Move up on to log4j2
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test

This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.

Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:21:33 -08:00
Jan Hentschel d50ae03716 HBASE-19537 Removed unnecessary semicolons from hbase-backup 2017-12-19 21:03:07 +01:00
Mike Drob 75f512bd71 HBASE-18838 Fix hadoop3 check-shaded-invariants 2017-12-15 11:19:47 -06:00
Josh Elser 4a1c3b4210 HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-12-14 15:11:19 -05:00
BELUGA BEHR 536187446d HBASE-19472 Remove ArrayUtil Class
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-13 19:02:07 +08:00
Peter Somogyi 7e5f3a516c HBASE-19213 Align check and mutate operations in Table and AsyncTable
- Deprecates old checkAnd*() operations in Table
- Adds Table#CheckAndMutateBuilder and implements it in HTable

Commiter note: When committing the patch, noticed redundant {@inheritDoc} being added in HTable.
Removed new and olds ones.
2017-12-12 20:37:24 -08:00
tedyu 82e278be55 HBASE-19469 Review Of BackupSystemTable (BELUGA BEHR) 2017-12-09 09:38:00 -08:00
tedyu 27ed4d8add HBASE-19417 Remove boolean return value from postBulkLoadHFile hook 2017-12-06 07:06:28 -08:00
tedyu 4c567b3c12 HBASE-19159 Backup should check permission for snapshot copy in advance (Janos Gub) 2017-12-04 11:59:18 -08:00
Michael Stack 5b13b624bb
HBASE-19181 LogRollBackupSubprocedure will fail if we use AsyncFSWAL instead of FSHLog (Vladimir Rodionov) 2017-11-16 15:04:07 -08:00
Tamas Penzes 377174d3ef HBASE-18601: Update Htrace to 4.2
Updated HTrace version to 4.2
Created TraceUtil class to wrap htrace methods. Uses try with resources.

Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-11 10:34:03 -08:00
tedyu cffbe4bef7 HBASE-19211 B&R: update configuration string in BackupRestoreConstants (Vladimir Rodionov) 2017-11-08 14:16:13 -08:00
tedyu a5a38b7916 HBASE-19211 B&R: update configuration string in BackupRestoreConstants (Vladimir Rodionov) 2017-11-08 14:12:43 -08:00
tedyu 6712f8f632 HBASE-18870 Hbase Backup should set the details to MR job name (Vishal Khandelwal) 2017-10-30 10:35:52 -07:00
Apekshit Sharma dd70cc3081 HBASE-19073 Cleanup CoordinatedStateManager
- Merged BaseCSM class into CSM interface
- Removed config hbase.coordinated.state.manager.class
- Since state manager is not pluggable anymore, we don't need start/stop/initialize to setup unknown classes. Our internal ZkCSM now requires Server in constructor itself. Makes the dependency clearer too.
- Removed CSM from HRegionServer and HMaster constructor. Although it's a step back from dependency injection, but it's more consistent with our current (not good)  pattern where we initialize everything in the ctor itself.

Change-Id: Ifca06bb354adec5b11ea1bad4707e014410491fc
2017-10-24 19:56:04 -07:00
Ramkrishna 70f4c5da47 HBSE-18945 Make a IA.LimitedPrivate interface for CellComparator (Ram) 2017-10-17 23:17:07 +05:30
tedyu a6c9d371df HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov ) 2017-10-13 15:04:53 -07:00
tedyu c4ced0b3d5 HBASE-18975 Fix backup / restore hadoop3 incompatibility (Vladimir Rodionov) 2017-10-11 12:26:34 -07:00
Amit 1ec6ece918 HBASE-18941 Confusing logging error around rerun of restore on an existing table.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-04 11:04:18 -07:00
Amit 2f2513255d HBASE-18932 Backup masking exception in a scenario and though it fails , it shows success message
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-04 08:18:08 -07:00
Ramkrishna 0a24178d06 HBASE-18649 Deprecate KV Usage in MR to move to Cells in 3.0 (ram) 2017-10-04 16:00:28 +05:30
tedyu 56830c3094 HBASE-18928 Backup delete command shows wrong number of deletes requested (Amit Kabra) 2017-10-03 14:07:58 -07:00
tedyu 1ff90d3b4b HBASE-18929 Hbase backup command doesn’t show debug option to enable backup in debug mode (Amit Kabra) 2017-10-03 10:11:27 -07:00
tedyu ca2959824d HBASE-18887 After full backup passed on hdfs root and incremental failed, full backup cannot be cleaned (Vladimir Rodionov) 2017-09-28 10:20:02 -07:00
Chia-Ping Tsai a11a35a113 HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:29 +08:00
Apekshit Sharma 2bf5bbd0a2 Revert HBASE-18843. Needs more work. Also, breaks building with Hadoop3. "Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov)"
This reverts commit 417dad646b.
2017-09-27 20:39:15 -07:00
Apekshit Sharma 97513466c0 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:40:25 -07:00
tedyu 417dad646b HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov) 2017-09-26 20:58:16 -07:00