Commit Graph

14459 Commits

Author SHA1 Message Date
Chia-Ping Tsai 9f1bfbeaab HBASE-18753 Introduce the unsynchronized TimeRangeTracker 2017-10-01 21:45:03 +08:00
Chia-Ping Tsai dd3d7de018 HBASE-13844 (Addendum) Replace KeyValue#isDelete by CellUtil#isDelete in hbase-shell 2017-10-01 17:47:47 +08:00
libisthanks 4aadc5d322 HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-30 02:08:42 -07:00
Ashu Pachauri 367dfabf06 Revert "HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region" due to wrong jira id.
This reverts commit f20580a530.
2017-09-30 01:44:39 -07:00
Apekshit Sharma cacf3f5835 HBASE-18884 (addendum) Add more javadoc comment. Coprocessor Design Improvements follow up of HBASE-17732.
Change-Id: Id1a9e508cc04612b1e79e6cfa7e39b7755598be3
2017-09-29 17:32:11 -07:00
Robert Yokota c835dcc7e7 HBASE-18559 Add histogram to MetricsConnection to track concurrent calls per server
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-29 17:12:26 -07:00
libisthanks f20580a530 HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-29 16:12:39 -07:00
Andrew Purtell ca87d05a51 HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 2017-09-29 15:37:04 -07:00
Biju Nair 3bd824faca HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-29 15:31:10 -07:00
Misty Stanley-Jones b0e1a15092 Update Misty's timezone
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-29 10:37:45 -07:00
Mike Drob 4136ab338d HBASE-18883 Update Curator to 4.0
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-29 10:39:57 -05:00
zhangduo 239e687267 HBASE-18845 TestReplicationSmallTests fails after HBASE-14004 2017-09-29 14:32:26 +08:00
Andy Yang afce850cfd HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 11:50:23 +08:00
Michael Stack b5b4108fce HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; MINOR AMENDMENT adding README to design-doc dir 2017-09-28 13:24:40 -07:00
Apekshit Sharma 74d0adce61 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:25:47 -07:00
tedyu ca2959824d HBASE-18887 After full backup passed on hdfs root and incremental failed, full backup cannot be cleaned (Vladimir Rodionov) 2017-09-28 10:20:02 -07:00
Chia-Ping Tsai a11a35a113 HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:29 +08:00
zhangduo 7f4c3b3564 HBASE-18826 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface 2017-09-28 15:26:52 +08:00
Ramkrishna 0cf15fadd8 HBASE-18888 StealJobQueue should call super() to init the
PriorityBlockingQueue (Ram)
2017-09-28 10:37:23 +05:30
Apekshit Sharma 2bf5bbd0a2 Revert HBASE-18843. Needs more work. Also, breaks building with Hadoop3. "Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov)"
This reverts commit 417dad646b.
2017-09-27 20:39:15 -07:00
Apekshit Sharma 97513466c0 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:40:25 -07:00
anoopsamjohn bd68551f26 HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:01:51 +05:30
tedyu 417dad646b HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov) 2017-09-26 20:58:16 -07:00
tedyu 976ef508f7 HBASE-18843 Add DistCp support to incremental backup with bulk loading - revert due to missing credit 2017-09-26 20:57:39 -07:00
tedyu 5df2fe7063 HBASE-18843 Add DistCp support to incremental backup with bulk loading 2017-09-26 20:55:19 -07:00
Guangxu Cheng 845b83b8b6 HBASE-18880 Failed to start rest server if the value of hbase.rest.threads.max is too small.
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-26 20:38:53 -07:00
shaofengshi 843917c525 HBASE-18885 HFileOutputFormat2 hardcodes default FileOutputCommitter
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 19:58:11 -07:00
Andrew Purtell 91e1f834bf Amend HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Adjust exception control flow to fix findbugs warning
NP_NULL_ON_SOME_PATH_EXCEPTION, Possible null pointer dereference of
regionSink in org.apache.hadoop.hbase.tool.Canary$RegionMonitor.run()
on exception path
2017-09-26 10:38:17 -07:00
Pankaj Kumar 9e7b16b88e HBASE-18874, HMaster abort message will be skipped if Throwable is passed null
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 08:17:34 -07:00
Michael Stack 410368b84e HBASE-18859 Purge PB from BulkLoadObserver
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-25 21:14:33 -07:00
Michael Stack 2ceeb54c92 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (Biju Nair); ADDENDUM 2017-09-25 20:57:58 -07:00
huzheng f54cc1ca51 HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-26 11:46:01 +08:00
anoopsamjohn 3c7ab81072 HBASE-16769 Deprecate/remove PB references from MasterObserver and RegionServerObserver 2017-09-26 08:52:23 +05:30
Chinmay Kulkarni 4b208eb2b6 HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Added assertion checks to make sure that the error code for the
ToolRunner run() method is used.

Testing Done: Checked that TestCanaryTool unit tests fail when there is
an error code in the current Canary run.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 19:06:36 -07:00
Chinmay Kulkarni 176571a852 HBASE-18762 Canary sink type cast error
Changed the type hierarchy of Canary sinks to reduce confusion and avoid
cast errors.

Testing Done: Ran the TestCanaryTool.java test suite and confirmed that
the working is correct.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 18:19:34 -07:00
Andrew Purtell b145286f36 Amend HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas
Remove now invalid unit test TestCorruptedRegionStoreFile
2017-09-25 17:36:04 -07:00
Guangxu Cheng cfb6a54f69 HBASE-18875 Thrift server supports read-only mode
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-25 15:11:03 -07:00
tedyu 0ebd4b8433 HBASE-18876 Backup create command fails to take queue parameter as option (Amit Kabra) 2017-09-25 12:57:54 -07:00
zhangduo a5f84430a3 HBASE-18825 Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface 2017-09-25 09:35:39 +08:00
Josh Elser 1540483135 HBASE-18807 Remove protobuf references from CP quota API calls 2017-09-23 22:06:51 -04:00
Sean Busbey b4830466db HBASE-18866 clean up warnings about proto syntax
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-22 18:36:08 -05:00
Andrew Purtell b27f9b582a HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas 2017-09-22 12:45:05 -07:00
Chia-Ping Tsai bb9f01c1e4 HBASE-18787 Fix the "dependencies connecting to an HBase cluster" 2017-09-23 01:56:38 +08:00
Abhishek Singh Chouhan 21a61ad597 HBASE-18796 Addendum: Scanner returning null incorrectly during locateRegionInMeta retry, resulting in incorrect TableNotFoundException
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-22 10:20:29 -07:00
Andrew Purtell 11d749527a HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4
Disable units where argument matching is intermittently failing
2017-09-22 09:48:17 -07:00
Reid Chan 5f238b3ef4 HBASE-18651 Let ChaosMonkeyRunner expose the chaos monkey runner it creates
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tedyu@apache.org>
2017-09-21 21:19:25 -05:00
Sean Busbey e39359986c HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-21 14:00:10 -05:00
Chia-Ping Tsai a6c3c645fd HBASE-18823 Apply RegionInfo to MasterObserver/RegionObserver/WALObserver 2017-09-20 18:43:11 -07:00
Michael Stack 37696fffe9 Revert "HBASE-16478 Rename WALKey in PB to WALEdit This is a rebase of Enis's original patch"
Not worth the difference it introduces; means hbase-protocol can no
longer parse a WAL entry.

This reverts commit 9a2e680cae.
2017-09-20 15:27:37 -07:00
Michael Stack e3896cfcc3 HBASE-18853 hbase-protocol-shaded includes protobuf (since we moved to hbase-thirdparty) 2017-09-20 08:15:01 -07:00