Commit Graph

14279 Commits

Author SHA1 Message Date
Apekshit Sharma e2ad4c038c HBASE-18931 Make ObserverContext an interface.
Change-Id: I9284a3271e06a3ee8ab9719cf012a4d8b3a82c88
2017-10-05 23:34:19 -07:00
Sean Busbey bafbade248 HBASE-18940 include project pylint configs in source artifact.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-05 10:38:28 -05:00
zhangduo 98d1637bcd HBASE-18878 Use Optional in return types.
These functions have been changed to return Optional<T> instead of T, where T = old return type.
- ObserverContext#getCaller
- RpcCallContext#getRequestUser
- RpcCallContext#getRequestUserName
- RpcServer#getCurrentCall
- RpcServer#getRequestUser
- RpcServer#getRequestUserName
- RpcServer#getRemoteAddress
- ServerCall#getRequestUser

Change-Id: Ib7b4e6be637283755f55755dd4c5124729f7052e
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-10-04 15:55:18 -07:00
Amit 1ec6ece918 HBASE-18941 Confusing logging error around rerun of restore on an existing table.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-04 11:04:18 -07:00
Chia-Ping Tsai 11aa6742f0 HBASE-18927 Add the DataType which is subset of KeyValue#Type to CellBuilder for building cell 2017-10-04 23:45:43 +08:00
Amit 2f2513255d HBASE-18932 Backup masking exception in a scenario and though it fails , it shows success message
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-04 08:18:08 -07:00
Ramkrishna 0a24178d06 HBASE-18649 Deprecate KV Usage in MR to move to Cells in 3.0 (ram) 2017-10-04 16:00:28 +05:30
Yi Liang 16d483f900 HBASE-16894 Create more than 1 split per region, generalize HBASE-12590
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-10-03 17:11:06 -07:00
tedyu 56830c3094 HBASE-18928 Backup delete command shows wrong number of deletes requested (Amit Kabra) 2017-10-03 14:07:58 -07:00
tedyu c205b5b5ed HBASE-18902 TestCoprocessorServiceBackwardCompatibility fails 2017-10-03 13:48:46 -07:00
Michael Stack 2ad7be24a9
HBASE-18606 Tests in hbase-spark module fail with UnsatisfiedLinkError 2017-10-03 12:13:24 -07:00
tedyu 1ff90d3b4b HBASE-18929 Hbase backup command doesn’t show debug option to enable backup in debug mode (Amit Kabra) 2017-10-03 10:11:27 -07:00
Peter Somogyi 0af61dce65
HBASE-18815 We need to pass something like CompactionRequest in CP to give user some information about the compaction
CompactionRequest was removed from CP in HBASE-18453, this change reintroduces
CompatcionRequest to CP as a read-only interface called CompactionRequest.
The CompactionRequest class is renamed to CompactionRequestImpl.

Additionally, this change removes selectionTimeInNanos from CompactionRequest and
uses selectionTime as a replacement. This means that CompactionRequest:toString
is modified and compare as well.

Signed-off-by: Michael Stack <stack@apache.org>
2017-10-03 07:52:33 -07:00
Ramkrishna 50265395d1 HBASE-18490 Modifying a table descriptor to enable replicas does not
create replica regions (Ram)
2017-10-03 13:10:44 +05:30
Guanghao Zhang afa03a207e HBASE-18913 TestShell fails because NoMethodError: undefined method parseColumn 2017-10-03 11:47:14 +08:00
libisthanks 214d21994e HBASE-18814 Make ScanMetrics enabled and add counter <HBase Counters, ROWS_SCANNED> into the MapReduce Job over snapshot
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-10-02 18:25:18 -07:00
Yi Liang 4093cc0291 HBASE-18894: null pointer exception in list_regions in shell command
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-02 16:50:29 -05:00
Michael Stack 38eaf47fa7 HBASE-18105 [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world (Yi Liang) 2017-10-02 11:38:11 -07:00
Chia-Ping Tsai d35d8376a7 HBASE-18897 Substitute MemStore for Memstore 2017-10-02 20:55:06 +08:00
Josh Elser 869b90c612 HBASE-17441 Fix invalid quoting around hadoop-3 build in yetus personality 2017-10-01 15:46:24 -04:00
Chia-Ping Tsai 9f1bfbeaab HBASE-18753 Introduce the unsynchronized TimeRangeTracker 2017-10-01 21:45:03 +08:00
Chia-Ping Tsai dd3d7de018 HBASE-13844 (Addendum) Replace KeyValue#isDelete by CellUtil#isDelete in hbase-shell 2017-10-01 17:47:47 +08:00
libisthanks 4aadc5d322 HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-30 02:08:42 -07:00
Ashu Pachauri 367dfabf06 Revert "HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region" due to wrong jira id.
This reverts commit f20580a530.
2017-09-30 01:44:39 -07:00
Apekshit Sharma cacf3f5835 HBASE-18884 (addendum) Add more javadoc comment. Coprocessor Design Improvements follow up of HBASE-17732.
Change-Id: Id1a9e508cc04612b1e79e6cfa7e39b7755598be3
2017-09-29 17:32:11 -07:00
Robert Yokota c835dcc7e7 HBASE-18559 Add histogram to MetricsConnection to track concurrent calls per server
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-29 17:12:26 -07:00
libisthanks f20580a530 HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-29 16:12:39 -07:00
Andrew Purtell ca87d05a51 HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 2017-09-29 15:37:04 -07:00
Biju Nair 3bd824faca HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-29 15:31:10 -07:00
Misty Stanley-Jones b0e1a15092 Update Misty's timezone
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-29 10:37:45 -07:00
Mike Drob 4136ab338d HBASE-18883 Update Curator to 4.0
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-29 10:39:57 -05:00
zhangduo 239e687267 HBASE-18845 TestReplicationSmallTests fails after HBASE-14004 2017-09-29 14:32:26 +08:00
Andy Yang afce850cfd HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 11:50:23 +08:00
Michael Stack b5b4108fce HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; MINOR AMENDMENT adding README to design-doc dir 2017-09-28 13:24:40 -07:00
Apekshit Sharma 74d0adce61 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:25:47 -07:00
tedyu ca2959824d HBASE-18887 After full backup passed on hdfs root and incremental failed, full backup cannot be cleaned (Vladimir Rodionov) 2017-09-28 10:20:02 -07:00
Chia-Ping Tsai a11a35a113 HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:29 +08:00
zhangduo 7f4c3b3564 HBASE-18826 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface 2017-09-28 15:26:52 +08:00
Ramkrishna 0cf15fadd8 HBASE-18888 StealJobQueue should call super() to init the
PriorityBlockingQueue (Ram)
2017-09-28 10:37:23 +05:30
Apekshit Sharma 2bf5bbd0a2 Revert HBASE-18843. Needs more work. Also, breaks building with Hadoop3. "Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov)"
This reverts commit 417dad646b.
2017-09-27 20:39:15 -07:00
Apekshit Sharma 97513466c0 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:40:25 -07:00
anoopsamjohn bd68551f26 HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:01:51 +05:30
tedyu 417dad646b HBASE-18843 Add DistCp support to incremental backup with bulk loading (Vladimir Rodionov) 2017-09-26 20:58:16 -07:00
tedyu 976ef508f7 HBASE-18843 Add DistCp support to incremental backup with bulk loading - revert due to missing credit 2017-09-26 20:57:39 -07:00
tedyu 5df2fe7063 HBASE-18843 Add DistCp support to incremental backup with bulk loading 2017-09-26 20:55:19 -07:00
Guangxu Cheng 845b83b8b6 HBASE-18880 Failed to start rest server if the value of hbase.rest.threads.max is too small.
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-26 20:38:53 -07:00
shaofengshi 843917c525 HBASE-18885 HFileOutputFormat2 hardcodes default FileOutputCommitter
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 19:58:11 -07:00
Andrew Purtell 91e1f834bf Amend HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Adjust exception control flow to fix findbugs warning
NP_NULL_ON_SOME_PATH_EXCEPTION, Possible null pointer dereference of
regionSink in org.apache.hadoop.hbase.tool.Canary$RegionMonitor.run()
on exception path
2017-09-26 10:38:17 -07:00
Pankaj Kumar 9e7b16b88e HBASE-18874, HMaster abort message will be skipped if Throwable is passed null
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 08:17:34 -07:00
Michael Stack 410368b84e HBASE-18859 Purge PB from BulkLoadObserver
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-25 21:14:33 -07:00