Commit Graph

14004 Commits

Author SHA1 Message Date
Ramkrishna 66fb60d4a4 HBASE-18649 Deprecate KV Usage in MR to move to Cells in 3.0 (Ram) 2017-10-04 14:44:19 +05:30
Yi Liang 4475ba88c1 HBASE-16894 Create more than 1 split per region, generalize HBASE-12590
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-10-03 17:15:39 -07:00
tedyu 7b8969412e HBASE-18928 Backup delete command shows wrong number of deletes requested (Amit Kabra) 2017-10-03 14:08:28 -07:00
tedyu 73414ab270 HBASE-18902 TestCoprocessorServiceBackwardCompatibility fails 2017-10-03 13:49:14 -07:00
Michael Stack 07b187bc73
HBASE-18606 Tests in hbase-spark module fail with UnsatisfiedLinkError 2017-10-03 12:12:47 -07:00
tedyu c4497c37d5 HBASE-18929 Hbase backup command doesn’t show debug option to enable backup in debug mode (Amit Kabra) 2017-10-03 10:11:58 -07:00
Peter Somogyi d3a817f212
HBASE-18815 We need to pass something like CompactionRequest in CP to give user some information about the compaction
CompactionRequest was removed from CP in HBASE-18453, this change reintroduces
CompatcionRequest to CP as a read-only interface called CompactionRequest.
The CompactionRequest class is renamed to CompactionRequestImpl.

Additionally, this change removes selectionTimeInNanos from CompactionRequest and
uses selectionTime as a replacement. This means that CompactionRequest:toString
is modified and compare as well.

Signed-off-by: Michael Stack <stack@apache.org>
2017-10-03 07:53:32 -07:00
Ramkrishna 3318e8724e HBASE-18490 Modifying a table descriptor to enable replicas does not
create replica regions (Ram)
2017-10-03 13:16:00 +05:30
Guanghao Zhang 2348b7fe45 HBASE-18913 TestShell fails because NoMethodError: undefined method parseColumn 2017-10-03 11:52:46 +08:00
libisthanks 0db82daa91 HBASE-18814 Make ScanMetrics enabled and add counter <HBase Counters, ROWS_SCANNED> into the MapReduce Job over snapshot
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-10-02 18:25:34 -07:00
Yi Liang 176d8bd2c7 HBASE-18894: null pointer exception in list_regions in shell command
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-02 16:51:34 -05:00
Michael Stack 24e40f3e85 HBASE-18105 [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world (Yi Liang) 2017-10-02 11:38:57 -07:00
Chia-Ping Tsai e047f518ef HBASE-18897 Substitute MemStore for Memstore 2017-10-02 21:10:11 +08:00
Josh Elser 8c6ed571bd HBASE-17441 Fix invalid quoting around hadoop-3 build in yetus personality 2017-10-01 15:21:54 -04:00
Chia-Ping Tsai 2cc1ea3d2e HBASE-18753 Introduce the unsynchronized TimeRangeTracker 2017-10-01 21:38:05 +08:00
anastas 6cf647a0df Merge branch 'branch-2' of https://git-wip-us.apache.org/repos/asf/hbase into branch-2 2017-10-01 16:23:57 +03:00
anastas 445c5bd7b7 HBASE-18010: CellChunkMap integration into CompactingMemStore, merge to Release 2.0, add new files 2017-10-01 16:22:34 +03:00
Chia-Ping Tsai f04bd9ca18 HBASE-13844 (Addendum) Replace KeyValue#isDelete by CellUtil#isDelete in hbase-shell 2017-10-01 17:54:23 +08:00
anastas 95405fbd83 HBASE-18010: CellChunkMap integration into CompactingMemStore, merge to Release 2.0 2017-10-01 11:01:10 +03:00
libisthanks 7ee44a820f HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-30 02:11:22 -07:00
Ashu Pachauri c256532ce3 Revert "HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region" due to wrong jira id.
This reverts commit 55987efdcf.
2017-09-30 02:00:16 -07:00
Apekshit Sharma 4579bba486 HBASE-18884 (addendum) Add more javadoc comment. Coprocessor Design Improvements follow up of HBASE-17732.
Change-Id: Id1a9e508cc04612b1e79e6cfa7e39b7755598be3
2017-09-29 17:32:41 -07:00
Robert Yokota 759110c225 HBASE-18559 Add histogram to MetricsConnection to track concurrent calls per server
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-29 17:12:26 -07:00
libisthanks 55987efdcf HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-29 16:21:51 -07:00
Biju Nair d142f07129 HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-29 16:19:39 -07:00
Andrew Purtell 5917ad4c0c HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 2017-09-29 15:36:42 -07:00
Mike Drob 5cbf158aaa HBASE-18883 Update Curator to 4.0
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-29 10:54:29 -05:00
zhangduo 2e4c1b6288 HBASE-18845 TestReplicationSmallTests fails after HBASE-14004 2017-09-29 14:32:33 +08:00
Andy Yang 0658252ed6 HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 13:20:24 +08:00
Michael Stack ac933f7af2 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; MINOR AMENDMENT adding README to design-doc dir 2017-09-28 13:25:07 -07:00
Apekshit Sharma a6a303816c HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:33:30 -07:00
tedyu a71f62de02 HBASE-18887 After full backup passed on hdfs root and incremental failed, full backup cannot be cleaned (Vladimir Rodionov) 2017-09-28 10:20:48 -07:00
Chia-Ping Tsai 6693f45faf HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:41 +08:00
zhangduo d26b8f8ddd HBASE-18826 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface 2017-09-28 15:41:56 +08:00
Ramkrishna f73a3a6fb7 HBASE-18888 StealJobQueue should call super() to init the
PriorityBlockingQueue (Ram)
2017-09-28 10:39:18 +05:30
Apekshit Sharma 75febcea89 Revert HBASE-18843. Needs more work. Also, breaks building with Hadoop3. "Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov)"
This reverts commit 447b591b08.
2017-09-27 20:44:30 -07:00
Apekshit Sharma 0c883a23c5 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:45:51 -07:00
anoopsamjohn 0fcc84cadd HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:02:57 +05:30
tedyu 447b591b08 HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov) 2017-09-26 20:59:12 -07:00
Guangxu Cheng 7e9534746c HBASE-18880 Failed to start rest server if the value of hbase.rest.threads.max is too small.
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-26 20:39:27 -07:00
shaofengshi e97738f0c2 HBASE-18885 HFileOutputFormat2 hardcodes default FileOutputCommitter
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 19:58:51 -07:00
Andrew Purtell ede916af5a Amend HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Adjust exception control flow to fix findbugs warning
NP_NULL_ON_SOME_PATH_EXCEPTION, Possible null pointer dereference of
regionSink in org.apache.hadoop.hbase.tool.Canary$RegionMonitor.run()
on exception path
2017-09-26 10:38:17 -07:00
Pankaj Kumar 6d0eb0eef0 HBASE-18874, HMaster abort message will be skipped if Throwable is passed null
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 08:17:58 -07:00
Michael Stack 2fee188156 HBASE-18859 Purge PB from BulkLoadObserver
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-25 21:14:59 -07:00
anoopsamjohn b6863d867d HBASE-16769 Deprecate/remove PB references from MasterObserver and RegionServerObserver 2017-09-26 09:30:59 +05:30
Michael Stack a8e8cf6e16 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (Biju Nair); ADDENDUM 2017-09-25 20:58:30 -07:00
huzheng 7c2622baf7 HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-26 11:23:38 +08:00
Chinmay Kulkarni 3e1ceadc80 HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Added assertion checks to make sure that the error code for the
ToolRunner run() method is used.

Testing Done: Checked that TestCanaryTool unit tests fail when there is
an error code in the current Canary run.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 19:06:44 -07:00
Chinmay Kulkarni b565a21480 HBASE-18762 Canary sink type cast error
Changed the type hierarchy of Canary sinks to reduce confusion and avoid
cast errors.

Testing Done: Ran the TestCanaryTool.java test suite and confirmed that
the working is correct.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 18:19:42 -07:00
Andrew Purtell fa0b0934db Amend HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas
Remove now invalid unit test TestCorruptedRegionStoreFile
2017-09-25 17:36:41 -07:00