Commit Graph

14026 Commits

Author SHA1 Message Date
Andy Yang 0658252ed6 HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 13:20:24 +08:00
Michael Stack ac933f7af2 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; MINOR AMENDMENT adding README to design-doc dir 2017-09-28 13:25:07 -07:00
Apekshit Sharma a6a303816c HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:33:30 -07:00
tedyu a71f62de02 HBASE-18887 After full backup passed on hdfs root and incremental failed, full backup cannot be cleaned (Vladimir Rodionov) 2017-09-28 10:20:48 -07:00
Chia-Ping Tsai 6693f45faf HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:41 +08:00
zhangduo d26b8f8ddd HBASE-18826 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface 2017-09-28 15:41:56 +08:00
Ramkrishna f73a3a6fb7 HBASE-18888 StealJobQueue should call super() to init the
PriorityBlockingQueue (Ram)
2017-09-28 10:39:18 +05:30
Apekshit Sharma 75febcea89 Revert HBASE-18843. Needs more work. Also, breaks building with Hadoop3. "Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov)"
This reverts commit 447b591b08.
2017-09-27 20:44:30 -07:00
Apekshit Sharma 0c883a23c5 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:45:51 -07:00
anoopsamjohn 0fcc84cadd HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:02:57 +05:30
tedyu 447b591b08 HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov) 2017-09-26 20:59:12 -07:00
Guangxu Cheng 7e9534746c HBASE-18880 Failed to start rest server if the value of hbase.rest.threads.max is too small.
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-26 20:39:27 -07:00
shaofengshi e97738f0c2 HBASE-18885 HFileOutputFormat2 hardcodes default FileOutputCommitter
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 19:58:51 -07:00
Andrew Purtell ede916af5a Amend HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Adjust exception control flow to fix findbugs warning
NP_NULL_ON_SOME_PATH_EXCEPTION, Possible null pointer dereference of
regionSink in org.apache.hadoop.hbase.tool.Canary$RegionMonitor.run()
on exception path
2017-09-26 10:38:17 -07:00
Pankaj Kumar 6d0eb0eef0 HBASE-18874, HMaster abort message will be skipped if Throwable is passed null
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 08:17:58 -07:00
Michael Stack 2fee188156 HBASE-18859 Purge PB from BulkLoadObserver
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-25 21:14:59 -07:00
anoopsamjohn b6863d867d HBASE-16769 Deprecate/remove PB references from MasterObserver and RegionServerObserver 2017-09-26 09:30:59 +05:30
Michael Stack a8e8cf6e16 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (Biju Nair); ADDENDUM 2017-09-25 20:58:30 -07:00
huzheng 7c2622baf7 HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-26 11:23:38 +08:00
Chinmay Kulkarni 3e1ceadc80 HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Added assertion checks to make sure that the error code for the
ToolRunner run() method is used.

Testing Done: Checked that TestCanaryTool unit tests fail when there is
an error code in the current Canary run.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 19:06:44 -07:00
Chinmay Kulkarni b565a21480 HBASE-18762 Canary sink type cast error
Changed the type hierarchy of Canary sinks to reduce confusion and avoid
cast errors.

Testing Done: Ran the TestCanaryTool.java test suite and confirmed that
the working is correct.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 18:19:42 -07:00
Andrew Purtell fa0b0934db Amend HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas
Remove now invalid unit test TestCorruptedRegionStoreFile
2017-09-25 17:36:41 -07:00
Guangxu Cheng bb81e9f3ca HBASE-18875 Thrift server supports read-only mode
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-25 15:11:23 -07:00
tedyu ad60bc5f60 HBASE-18876 Backup create command fails to take queue parameter as option (Amit Kabra) 2017-09-25 12:58:32 -07:00
zhangduo 06d8a23531 HBASE-18825 Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface 2017-09-25 09:48:33 +08:00
Josh Elser 40a73c9be5 HBASE-18807 Remove protobuf references from CP quota API calls 2017-09-23 21:28:58 -04:00
Sean Busbey 27646693f4 HBASE-18866 clean up warnings about proto syntax
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-22 18:38:11 -05:00
Andrew Purtell 60741ba0a4 HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas 2017-09-22 12:40:34 -07:00
Abhishek Singh Chouhan 8df523bdd0 HBASE-18796 Addendum: Scanner returning null incorrectly during locateRegionInMeta retry, resulting in incorrect TableNotFoundException
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-22 10:21:06 -07:00
Andrew Purtell a9fcd95860 HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4
Disable units where argument matching is intermittently failing
2017-09-22 09:48:02 -07:00
Reid Chan 3a02a2a80d HBASE-18651 Let ChaosMonkeyRunner expose the chaos monkey runner it creates
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tedyu@apache.org>
2017-09-21 21:34:07 -05:00
Sean Busbey c1f5122fab HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-21 14:01:02 -05:00
Chia-Ping Tsai 0eab16fde4 HBASE-18823 Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
Change-Id: I3f2faab96bf3aa3e033b9472e09cd0becf2e272c
2017-09-20 18:49:46 -07:00
Michael Stack e294dcd80d Revert "HBASE-16478 Rename WALKey in PB to WALEdit This is a rebase of Enis's original patch"
Not worth the difference it introduces; means hbase-protocol can no
longer parse a WAL entry.

This reverts commit 9a2e680cae.
2017-09-20 15:36:47 -07:00
Michael Stack 76199a338d HBASE-18853 hbase-protocol-shaded includes protobuf (since we moved to hbase-thirdparty) 2017-09-20 08:15:31 -07:00
Sean Busbey ea610cfcf7 HBASE-18833 Ensure precommit personality is up to date on all active branches
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-09-20 05:51:42 -05:00
Zach York d0a54352e8 HBASE-18478 Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
This provides significant cluster start time reduction for FileSystems which do not surface locality (S3).

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-19 16:31:00 -07:00
Abhishek Singh Chouhan 518674bce6 HBASE-18796 Admin#isTableAvailable returns incorrect result before daughter regions are opened
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-19 15:06:15 -07:00
Josh Elser 1db463feba HBASE-18851 Add "BSD" to the acceptable licenses list
org.ow2.asm:asm actually is 3-clause BSD, but it was
marked as "BSD" which we need to add to our exclusions
list.
2017-09-19 16:32:15 -04:00
Yu Li 52c809c35e HBASE-15931 Add log for long-running tasks in AsyncProcess (addendum) 2017-09-19 09:55:11 +08:00
Umesh Agashe 00d9e38479
HBASE-18832 Fixed LTT, removed references to deprecated HColumnDescriptor
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-18 13:10:07 -07:00
Yi Liang a66f9dcda5
HBASE-18803: Mapreduce job get failed caused by NoClassDefFoundError: org/apache/commons/lang3/ArrayUtils
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-18 12:36:05 -07:00
Reid Chan 2903938153 HBASE-18808 Ineffective config check in BackupLogCleaner#getDeletableFiles()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-18 10:17:00 -07:00
Chun-Hao Tang 03eefa8aa5 HBASE-18142 Deletion of a cell deletes the previous versions too
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-19 00:07:22 +08:00
Sean Busbey 83e9188d0c HBASE-18836 Note need for explicit javax.el and exclude from shaded artifacts
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2017-09-18 09:53:55 -05:00
Chia-Ping Tsai f380676d49 HBASE-18835 The return type of ExtendedCell#deepClone should be ExtendedCell rather than Cell 2017-09-18 22:38:42 +08:00
Sean Busbey 5ed2f899e1 HBASE-18834 fix shellcheck warning on hbase personality
Signed-off-by: Dima Spivak <dimaspivak@apache.org>
2017-09-18 08:55:30 -05:00
Chia-Ping Tsai 2aa10cb651 HBASE-18798 Remove the unused methods in RegionServerObserver 2017-09-18 09:49:38 +08:00
Michael Stack a1b6dd22e9 HBASE-18831 Add explicit dependency on javax.el
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-16 10:38:17 -07:00
Chia-Ping Tsai 3f112ee320 HBASE-17980 (Addendum) make UnmodifyableHRegionInfo deprecated 2017-09-16 17:08:34 +08:00