Commit Graph

6141 Commits

Author SHA1 Message Date
Peter Somogyi 0af61dce65
HBASE-18815 We need to pass something like CompactionRequest in CP to give user some information about the compaction
CompactionRequest was removed from CP in HBASE-18453, this change reintroduces
CompatcionRequest to CP as a read-only interface called CompactionRequest.
The CompactionRequest class is renamed to CompactionRequestImpl.

Additionally, this change removes selectionTimeInNanos from CompactionRequest and
uses selectionTime as a replacement. This means that CompactionRequest:toString
is modified and compare as well.

Signed-off-by: Michael Stack <stack@apache.org>
2017-10-03 07:52:33 -07:00
Ramkrishna 50265395d1 HBASE-18490 Modifying a table descriptor to enable replicas does not
create replica regions (Ram)
2017-10-03 13:10:44 +05:30
libisthanks 214d21994e HBASE-18814 Make ScanMetrics enabled and add counter <HBase Counters, ROWS_SCANNED> into the MapReduce Job over snapshot
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-10-02 18:25:18 -07:00
Michael Stack 38eaf47fa7 HBASE-18105 [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world (Yi Liang) 2017-10-02 11:38:11 -07:00
Chia-Ping Tsai d35d8376a7 HBASE-18897 Substitute MemStore for Memstore 2017-10-02 20:55:06 +08:00
Chia-Ping Tsai 9f1bfbeaab HBASE-18753 Introduce the unsynchronized TimeRangeTracker 2017-10-01 21:45:03 +08:00
libisthanks 4aadc5d322 HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-30 02:08:42 -07:00
Ashu Pachauri 367dfabf06 Revert "HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region" due to wrong jira id.
This reverts commit f20580a530.
2017-09-30 01:44:39 -07:00
libisthanks f20580a530 HBASE-18814 Improve TableSnapshotInputFormat to allow more multiple mappers per region
Signed-off-by: Ashu Pachauri <ashu@apache.org>
2017-09-29 16:12:39 -07:00
Andrew Purtell ca87d05a51 HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 2017-09-29 15:37:04 -07:00
zhangduo 239e687267 HBASE-18845 TestReplicationSmallTests fails after HBASE-14004 2017-09-29 14:32:26 +08:00
Andy Yang afce850cfd HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 11:50:23 +08:00
Apekshit Sharma 74d0adce61 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:25:47 -07:00
Chia-Ping Tsai a11a35a113 HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:29 +08:00
zhangduo 7f4c3b3564 HBASE-18826 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface 2017-09-28 15:26:52 +08:00
Ramkrishna 0cf15fadd8 HBASE-18888 StealJobQueue should call super() to init the
PriorityBlockingQueue (Ram)
2017-09-28 10:37:23 +05:30
Apekshit Sharma 97513466c0 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:40:25 -07:00
anoopsamjohn bd68551f26 HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:01:51 +05:30
Andrew Purtell 91e1f834bf Amend HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Adjust exception control flow to fix findbugs warning
NP_NULL_ON_SOME_PATH_EXCEPTION, Possible null pointer dereference of
regionSink in org.apache.hadoop.hbase.tool.Canary$RegionMonitor.run()
on exception path
2017-09-26 10:38:17 -07:00
Pankaj Kumar 9e7b16b88e HBASE-18874, HMaster abort message will be skipped if Throwable is passed null
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-26 08:17:34 -07:00
Michael Stack 410368b84e HBASE-18859 Purge PB from BulkLoadObserver
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-25 21:14:33 -07:00
Michael Stack 2ceeb54c92 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (Biju Nair); ADDENDUM 2017-09-25 20:57:58 -07:00
huzheng f54cc1ca51 HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-09-26 11:46:01 +08:00
anoopsamjohn 3c7ab81072 HBASE-16769 Deprecate/remove PB references from MasterObserver and RegionServerObserver 2017-09-26 08:52:23 +05:30
Chinmay Kulkarni 4b208eb2b6 HBASE-18830 TestCanaryTool does not check Canary monitor's error code
Added assertion checks to make sure that the error code for the
ToolRunner run() method is used.

Testing Done: Checked that TestCanaryTool unit tests fail when there is
an error code in the current Canary run.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 19:06:36 -07:00
Chinmay Kulkarni 176571a852 HBASE-18762 Canary sink type cast error
Changed the type hierarchy of Canary sinks to reduce confusion and avoid
cast errors.

Testing Done: Ran the TestCanaryTool.java test suite and confirmed that
the working is correct.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-25 18:19:34 -07:00
Andrew Purtell b145286f36 Amend HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas
Remove now invalid unit test TestCorruptedRegionStoreFile
2017-09-25 17:36:04 -07:00
zhangduo a5f84430a3 HBASE-18825 Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface 2017-09-25 09:35:39 +08:00
Josh Elser 1540483135 HBASE-18807 Remove protobuf references from CP quota API calls 2017-09-23 22:06:51 -04:00
Andrew Purtell b27f9b582a HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas 2017-09-22 12:45:05 -07:00
Andrew Purtell 11d749527a HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4
Disable units where argument matching is intermittently failing
2017-09-22 09:48:17 -07:00
Chia-Ping Tsai a6c3c645fd HBASE-18823 Apply RegionInfo to MasterObserver/RegionObserver/WALObserver 2017-09-20 18:43:11 -07:00
Michael Stack 37696fffe9 Revert "HBASE-16478 Rename WALKey in PB to WALEdit This is a rebase of Enis's original patch"
Not worth the difference it introduces; means hbase-protocol can no
longer parse a WAL entry.

This reverts commit 9a2e680cae.
2017-09-20 15:27:37 -07:00
Zach York 8301734594 HBASE-18478 Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
This provides significant cluster start time reduction for FileSystems which do not surface locality (S3).

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-19 17:35:04 -07:00
Abhishek Singh Chouhan 29a3ff3037 HBASE-18796 Admin#isTableAvailable returns incorrect result before daughter regions are opened
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-19 15:06:15 -07:00
Sean Busbey 5264f04b6d HBASE-18836 Note need for explicit javax.el and exclude from shaded artifacts
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2017-09-18 09:52:10 -05:00
Sean Busbey 2a8fb7a9ae Revert "HBASE-18836 Note need for explicit javax.el and exclude from shaded artifacts"
This reverts commit de32b11fe4.

Mistaken push of wrong branch.
2017-09-18 08:53:42 -05:00
Sean Busbey de32b11fe4 HBASE-18836 Note need for explicit javax.el and exclude from shaded artifacts 2017-09-18 08:47:46 -05:00
Chia-Ping Tsai 190a660d17 HBASE-18798 Remove the unused methods in RegionServerObserver 2017-09-18 09:44:57 +08:00
Michael Stack 5c12e424e6 HBASE-18831 Add explicit dependency on javax.el
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-16 10:37:27 -07:00
zhangduo a6d8cedb06 HBASE-18446 Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix) 2017-09-15 20:36:46 +08:00
zhangduo 4341c3f554 HBASE-14004 [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin 2017-09-15 19:22:00 +08:00
Yechao Chen 5c07dba43a HBASE-18772 (Addendum) we pass the wrong arguments to AdaptiveLifoCoDelCallQueue
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-15 10:27:47 +08:00
Chia-Ping Tsai 9c5b03acd7 HBASE-18766 Make TableSnapshotScanner Audience Private
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-14 14:30:05 -07:00
brandboat 58988cb53c HBASE-17980 Any HRegionInfo we give out should be immutable
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-14 14:25:07 -07:00
Michael Stack 98f4171109 HBASE-18818 TestConnectionImplemenation fails 2017-09-14 12:27:37 -07:00
Peter Somogyi 778f507ada HBASE-18683 Upgrade hbase to commons-math 3
Upgrade commons-math:2.2 to commons-math3:3.6.1
Remove commons-math 2 specific content from LICENSE.vm
Add missing jersey-client dependency to hbase-it module

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-14 08:34:28 -07:00
Reid Chan c17b491432 HBASE-18801 Bulk load cleanup may falsely deem file deletion successful
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-14 08:09:42 -07:00
zhangduo 61d10feffa HBASE-18453 CompactionRequest should not be exposed to user directly 2017-09-14 20:37:33 +08:00
Chia-Ping Tsai 38e983ed44 HBASE-18794 Remove deprecated methods in MasterObserver 2017-09-14 15:10:20 +08:00
Michael Stack d2792a07fe HBASE-18772 [JDK8] Replace AtomicLong with LongAdder; ADDENDUM -- disable broke test 2017-09-13 23:24:53 -07:00
Michael Stack 780be085ed HBASE-14998 Unify synchronous and asynchronous methods in Admin and cleanup
This is based on patch sent me by Balazs Meszaros. The good stuff in
here is from him. This patch does less than his ambition. It changes
Admin class only. Can work on making AsyncAdmin cohere in a follow-on.

 * Deprecates getAlterStatus. Everywhere else we talk of 'modify' rather
   'alter' and should use Future returned from async instead.
 * isTableAvailable(TableName, byte [][]) has been deprecated to be
   removed; use the overrie instead. This is a weird method.
 * Changed listTableDescriptor to getDescriptor.
 * Renamed other like methods to have same pattern (deprecating the old):
    balancer => balance
    setBalancerRunning => balancerSwitch
    setNormalizerRunning => normalizerSwitch
    enableCatalogJanitor => catalogJanitorSwitch
    setCleanerChoreRunning => cleanerChoreSwitch
    setSplitOrMergeEnabled => splitOrMergeEnabledSwitch

 * Renamed (with deprecation of old) runCatalogScan => runCatalogJanitor.
 * Reviewed generated javadoc and made some edits; purged reference to
   hbase issues from our API, fixed param names, etc.
 * Made all the enable services methods have same pattern.
 * Renamed takeSnapshotAsync as snapshotAsync (with deprecation of old)
 * Renamed execProcedureWithRet as execProcedureWithReturn (with
   deprecation)

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-13 17:19:25 -07:00
Michael Stack 8bbfc836db HBASE-18723 [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit; ADDENDUM3 2017-09-13 14:55:42 -07:00
Reid Chan 5d6b20ad18 HBASE-18609 Apply ClusterStatus#getClusterStatus(EnumSet<Option>) in code base
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-14 00:54:16 +08:00
Guangxu Cheng 93f5624cb9 HBASE-18789 Displays the reporting interval of each RS on the Master page
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-13 05:51:18 -07:00
zhangduo e6e52cd80f HBASE-18793 Remove deprecated methods in RegionObserver 2017-09-13 11:33:03 +08:00
Sean Busbey 4b124913f0 HBASE-17823 Migrate to Apache Yetus Audience Annotations
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-12 20:53:30 -05:00
Abhishek Singh Chouhan 3df0351f22 HBASE-18771 Incorrect StoreFileRefresh leading to split and compaction failures
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-12 17:45:15 -07:00
Yechao Chen eb5e43673c HBASE-18772 [JDK8] Replace AtomicLong with LongAdder
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-13 06:09:51 +08:00
Michael Stack a66bd04815 HBASE-18733 [compat 1-2] Hide WALKey 2017-09-12 13:12:14 -07:00
Guangxu Cheng cfdbdd2066 HBASE-18131 Add an hbase shell command to clear deadserver list in ServerManager
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-12 08:29:16 -07:00
Umesh Agashe 58bfa13075 HBASE-13271 Added test for batch operations with validation errors. Updated Javadoc for batch methods.
Javadoc for following methods are updated:
* Table.put(List<Put> puts)
* Table.delete(List<Delete> deletes)

Added @apiNote for delete regarding input list will not be modied in version 3.0.0

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-11 20:05:34 -07:00
Michael Stack d6db4a2d3d HBASE-16479 Move WALEdit from hbase.regionserver.wal package to hbase.wal package 2017-09-11 14:43:00 -07:00
Peter Somogyi cdc84fe818 HBASE-10240 Remove 0.94->0.96 migration code
Change-Id: I70cdbd1f1159ca923dc79221601e6190db2b4de1

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-11 12:07:46 -07:00
Yun Zhao 90c15bae8d HBASE-18662 The default values for many configuration items in the code are not consistent with hbase-default.xml
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-11 21:11:58 +08:00
Michael Stack 9a2e680cae HBASE-16478 Rename WALKey in PB to WALEdit This is a rebase of Enis's original patch 2017-09-10 21:58:51 -07:00
Chia-Ping Tsai 2359ef518a Revert "HBASE-18662 The default values for many configuration items in the code are not consistent with hbase-default.xml"
need more checks
This reverts commit 6035035707.
2017-09-09 22:54:38 +08:00
Yun Zhao 6035035707 HBASE-18662 The default values for many configuration items in the code are not consistent with hbase-default.xml
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-09 18:14:34 +08:00
Reid Chan 77ca743d09 HBASE-18621 Refactor ClusterOptions before applying to code base
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-09 03:31:28 +08:00
Biju Nair af21572427 HBASE-18641 Include block content verification logic used in lruCache in bucketCache
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-08 11:24:31 -07:00
Balazs Meszaros 359fed7b4b HBASE-18106 Redo ProcedureInfo and LockInfo
Main changes:
- ProcedureInfo and LockInfo were removed, we use JSON instead of them
- Procedure and LockedResource are their server side equivalent
- Procedure protobuf state_data became obsolate, it is only kept for
reading previously written WAL
- Procedure protobuf contains a state_message field, which stores the internal
state messages (Any type instead of bytes)
- Procedure.serializeStateData and deserializeStateData were changed slightly
- Procedures internal states are available on client side
- Procedures are displayed on web UI and in shell in the following jruby format:
  { ID => '1', PARENT_ID = '-1', PARAMETERS => [ ..extra state information.. ] }

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-08 10:24:04 -07:00
Chia-Ping Tsai 87c3ae9314 HBASE-18780 Remove HLogPrettyPrinter
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-08 10:20:20 -07:00
Michael Stack b8e0a8396f HBASE-18769 Make CompareFilter use generic CompareOperator instead of
internal enum
2017-09-08 10:06:56 -07:00
zhangduo 331910192a HBASE-18778 Use Comparator for StealJobQueue 2017-09-08 21:27:12 +08:00
Umesh Agashe 5847c901a7 HBASE-18543 [AMv2] Fixed and re-enabled TestMasterFailover
* testSimpleMasterFailover - fixed and verified
  * testPendingOpenOrCloseWhenMasterFailover - removed as logic is based on old code and no longer relevant. TestServerCrashProcedure tests assignments with crashing master and region servers
  * testMetaInTransitionWhenMasterFailover - verified that it is fixed by patch for HBASE-18511.

Signed-off-by: Michael Stack <stack@apache.org>
2017-09-07 16:45:52 -07:00
Apekshit Sharma 6752eba68f HBASE-14997 Move compareOp and Comparators out of filter to client package
Add a new generic CompareOperator enum and use it instead of the filter
CompareOp enum everywhere BUT inside CompareFilter.
2017-09-07 10:03:45 -07:00
Guangxu Cheng 5ff04c5e7f HBASE-17972 Remove mergePool from CompactSplitThread
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-07 09:39:53 -07:00
Peter Somogyi 137b105c67 HBASE-18704 Upgrade hbase to commons-collections 4
Upgrade commons-collections:3.2.2 to commons-collections4:4.1
Add missing dependency for hbase-procedure, hbase-thrift
Replace CircularFifoBuffer with CircularFifoQueue in WALProcedureStore and TaskMonitor

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-07 10:30:01 -05:00
tedyu df34300cd3 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-09-07 04:07:09 -07:00
Michael Stack 3a9dc8fbd5 HBASE-18768 Move TestTableName to hbase-common from hbase-server
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-06 18:11:32 -07:00
Chia-Ping Tsai 2f14a054f8 HBASE-18749 Apply the CF specific TimeRange from Scan to filter the segment scanner 2017-09-06 19:40:51 +08:00
Zach York f36fb11eaf HBASE-18757 Fix improper bitwise & in bucketcache offset calculation
This correctly casts the operand to a long to avoid negative offsets created by sign extending the integer operand.

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-05 14:11:54 -07:00
Biju Nair f362ef709f HBASE-18737 Display configured max size of memstore and cache on RS UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-05 13:01:11 -07:00
Umesh Agashe 5d60123f3e HBASE-18674 upgrading to commons-lang3
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-05 09:46:10 -07:00
Biju Nair 36371acee7 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-05 08:13:19 -07:00
Michael Stack 52d4b25754 HBASE-18723 [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit; ADDENDUM2
Plug more holes. REST can't find JspC when it runs its antrun task.
hbase-it didn't include reference to hbase-protocol-shaded
2017-09-04 21:13:36 -07:00
Michael Stack 8ec5d59a08 HBASE-18736 Cleanup the HTD/HCD for Admin; ADDENDUM 2017-09-04 21:09:04 -07:00
Chia-Ping Tsai 32ddb1f604 HBASE-18736 Cleanup the HTD/HCD for Admin 2017-09-04 16:59:36 +08:00
tedyu 83175fdf83 HBASE-18743 HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default table are deleted (wenbang) 2017-09-03 07:10:19 -07:00
zhangduo 9e53f2927b HBASE-18699 Copy LoadIncrementalHFiles to another package and mark the old one as deprecated 2017-09-03 19:49:42 +08:00
Michael Stack fb537fe736 HBASE-18723 [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit
Do a pass with dependency:analyze; remove unused and
explicity list the dependencies we exploit.
Remove the parent dependencies set which had junit, mockito,
log4j, and findbugs annotations (had to put junit back
temporarily in subsequent version of this patch TODO). Listing in
parent set meant these libs were dependencies for all modules
which in practice was not the case. Edited all modules so
those that need any from this parent set now do explicit listing.

Ran the dependency:analyze over the project. Acted on most
suggested removals and requests for explicit listing. Some
grey areas remain around transitives that come in with
hadoop -needs better excludes, another project- and that
the dependency:analyze tool is not always accurate in its
reporting.
2017-08-31 12:41:31 -07:00
Chia-Ping Tsai 8a800c3f19 HBASE-18721 Cleanup unused configs and private declaration 2017-08-31 23:40:37 +08:00
Cesar Delgado b5942241ca HBASE-18675 Making {max,min}SessionTimeout configurable for MiniZooKeeperCluster
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-08-30 14:14:40 -07:00
Sean Busbey 1e28c9be6b Revert "Making {max,min}SessionTimeout configurable for MiniZooKeeperCluster"
This reverts commit bcefdf7d18.
2017-08-30 14:13:55 -07:00
Cesar Delgado bcefdf7d18 Making {max,min}SessionTimeout configurable for MiniZooKeeperCluster
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-08-30 14:13:09 -07:00
Chia-Ping Tsai 7465973068 HBASE-15806 An endpoint-based export tool 2017-08-30 14:06:04 +08:00
Chia-Ping Tsai d374a01415 HBASE-18306 Get rid of TableDescriptor#getConfiguration 2017-08-30 11:23:12 +08:00
Sean Mackrory b3c5c5ed4e HBASE-15947 Classes used only for tests included in main code base
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-30 10:46:28 +08:00
Chia-Ping Tsai 899255b76c HBASE-18714 The dropBehind and readahead don't be applied when useHBaseChecksum is enabled 2017-08-30 10:26:34 +08:00
Michael Stack 8c9087e6c5 HBASE-18698 MapreduceDependencyClasspathTool does not include hbase-server as a dependency
Move Driver to be the main-class in hbase-mapreduce jar rather than
in the hbase-server jar.

Reference the hbase-server and shaded protobuf so they get bundled
when you do 'hbase mapredcp'.
2017-08-29 13:09:57 -07:00
Andy Yang c91af3e7a4 HBASE-3935 HServerLoad.storefileIndexSizeMB should be changed to storefileIndexSizeKB
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-29 13:11:00 +08:00
Michael Stack 47a5614edd Revert "HBASE-18674 upgrading to commons-lang3"
Premature push

This reverts commit 5dacc85122.
2017-08-28 16:41:34 -07:00
Umesh Agashe 5dacc85122 HBASE-18674 upgrading to commons-lang3 2017-08-28 15:57:12 -07:00
Tamas Penzes bbf23d9627 HBASE-18369 hbase thrift web-ui not available
classpath issue when running as a developer fixed
removed thrift webapp from server (it's not used at all, since moved to thrift webapp)

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-28 08:45:57 -07:00
Vikas Vishwakarma 4e7f677170 HBASE-18633 Add more info to understand the source/scenario of large batch requests exceeding threshold 2017-08-28 15:20:21 +05:30
Chia-Ping Tsai d63a287bfd HBASE-18519 Use builder pattern to create cell 2017-08-28 14:14:49 +08:00
Chun-Hao Tang a407b58f17 HBASE-18611 Copy all tests from o.a.h.h.p.TestProtobufUtil to o.a.h.h.s.p.TestProtobufUtil
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-27 19:16:06 +08:00
Apekshit Sharma 664b6be0ef HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
  on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
  These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
  This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
  confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
  more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
  big win!

Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-25 18:38:48 -07:00
zhangduo 8d33949b8d HBASE-16324 Remove LegacyScanQueryMatcher 2017-08-26 08:04:43 +08:00
Michael Stack 20d272bce5 HBASE-18687 Add @since 2.0.0 to new classes 2017-08-25 13:12:30 -07:00
Michael Stack 08a9522fc1 HBASE-18673 Some more unwanted reference to unshaded PB classes 2017-08-24 20:59:23 -07:00
Ajay Jadhav 98bb5c05e3 HBASE-18448 Added refresh HFiles coprocessor endpoint
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2017-08-25 06:56:41 +05:30
Huaxiang Sun 1a2c38b965 HBASE-18287 Remove log warning in PartitionedMobCompactor.java#getFileStatus 2017-08-24 14:43:44 -07:00
Michael Stack 1b4e935cec HBASE-18658 Purge hokey hbase Service implementation; use (internal) Guava Service instead
Removes hbase Service. Moves the single user, ClusterSchemaServiceImpl
to use relocated internal Guava Service instead.
2017-08-24 09:19:36 -07:00
anoopsamjohn 88356029f1 Revert "HBASE-18448 Added refresh HFiles coprocessor endpoint"
This reverts commit 612c23556d.
2017-08-24 21:13:08 +05:30
Ajay Jadhav 612c23556d HBASE-18448 Added refresh HFiles coprocessor endpoint
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2017-08-24 20:42:49 +05:30
Michael Stack 6e7baa07f0 HBASE-15982 Interface ReplicationEndpoint extends Guava's Service
Breaking change to our ReplicationEndpoint and BaseReplicationEndpoint.

ReplicationEndpoint implemented Guava 0.12 Service. An abstract
subclass, BaseReplicationEndpoint, provided default implementations
and facility, among other things, by extending Guava
AbstractService class.

Both of these HBase classes were marked LimitedPrivate for
REPLICATION so these classes were semi-public and made it so
Guava 0.12 was part of our API.

Having Guava in our API was a mistake. It anchors us and the
implementation of the Interface to Guava 0.12. This is untenable
given Guava changes and that the Service Interface in particular
has had extensive revamp and improvement done. We can't hold to
the Guava Interface. It changed. We can't stay on Guava 0.12;
implementors and others on our CLASSPATH won't abide being stuck
on an old Guava.

So this class makes breaking changes. The unhitching of our Interface
from Guava could only be done in a breaking manner. It undoes the
LimitedPrivate on BaseReplicationEndpoint while keeping it for the RE
Interface. It means consumers will have to copy/paste the
AbstractService-based BRE into their own codebase also supplying their
own Guava; HBase no longer 'supplies' this (our Guava usage has
been internalized, relocated).

This patch then adds into RE the basic methods RE needs of the old
Guava Service rather than return a Service to start/stop only to go
back to the RE instance to do actual work. A few method names had to
be changed so could make implementations with Guava Service internally
and not have RE method names and types clash). Semantics remained the
same otherwise. For example startAsync and stopAsync in Guava are start
and stop in RE.
2017-08-24 08:05:27 -07:00
zhangduo d12eb7a4aa HBASE-18347 Implement a BufferedMutator for async client 2017-08-24 15:55:43 +08:00
Chia-Ping Tsai 25ee5f7f84 HBASE-18546 Always overwrite the TS for Append/Increment unless no existing cells are found 2017-08-24 14:35:22 +08:00
Michael Stack 3b444a066c HBASE-18224 Upgrade Jetty 2017-08-23 23:12:59 -07:00
Chia-Ping Tsai 25ff9d0bbf HBASE-18503 Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor 2017-08-24 13:03:38 +08:00
Chia-Ping Tsai 12f2b02a80 Revert "HBASE-18503 Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor"
Wrong author information
This reverts commit b03348630c.
2017-08-24 13:02:11 +08:00
Chia-Ping Tsai b03348630c HBASE-18503 Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor 2017-08-24 12:56:58 +08:00
Apekshit Sharma 26e6c2ceb4 HBASE-17442 Move most of the replication related classes from hbase-client to new hbase-replication package. (Guanghao Zhang).
Change-Id: Ie0e24cc617ab4bf56de8b1747062d1b78a5d4669
2017-08-23 14:41:58 -07:00
Umesh Agashe 6b21f8881b HBASE-18575 [AMv2] Fixed and enabled TestRestartCluster#testRetainAssignmentOnRestart on master
* Fixed ServerCrashProcedure to set forceNewPlan to false for instances AssignProcedure. This enables balancer to find most suitable target server
* Fixed and enabled TestRestartCluster#testRetainAssignmentOnRestart on master
* Renamed method ServerName@isSameHostnameAndPort() to isSameAddress()

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-23 10:10:56 -07:00
Vladimir Rodionov 2dda371209 HBASE-17614: Move Backup/Restore into separate module (Vladimir Rodionov)
Signed-off-by: Josh Elser <elserj@apache.org>
2017-08-23 12:40:57 -04:00
tedyu 6c0e219dd4 HBASE-18632 TestMultiParallel#testFlushCommitsWithAbort fails in master branch 2017-08-23 09:35:08 -07:00
Biju Nair 04f114b85c HBASE-18532 Improve cache related stats rendered on RS UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-23 07:53:55 -07:00
Michael Stack dcd3e9abf4 HBASE-18655 TestAsyncClusterAdminApi2 failing sometimes
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-22 21:24:10 -07:00
Biju Nair 3be5e8ce00 HBASE-18614 Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-22 18:01:43 -07:00
Mike Drob 51d458872d HBASE-12349 Add custom error-prone module 2017-08-22 16:38:17 -05:00
Mike Drob 5b42d81950 HBASE-18628 Fix event pre-emption in ZKPermWatcher
Instead of using an Atomic Reference to data and aborting when we detect
that new data comes in, use the native cancellation/pre-emption features
of Java Future.
2017-08-22 11:19:38 -05:00
Michael Stack 05f07f692b HBASE-18645 Loads of tests timing out.... Revert "HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable"
This reverts commit 1ab6882f62.
2017-08-22 01:41:18 -07:00
Umesh Agashe 5895538a32 HBASE-18103 [AMv2] Changed master to throw YouAreDeadException when it receives regionServerReport() with incorrect region assignment. Added test to verify rogue region server behavior.
Behavior prior to these changes is to call expireServer(), log exception and suppress it. These changes will result in RS receiving the YouAreDeadException and treating it as a fatal error. This 'fail fast' approach will help us stabilize the code. This behavior can be reconsidered later if necessary.

Signed-off-by: Michael Stack <stack@apache.org>
2017-08-21 17:23:10 -07:00
Michael Stack bf343da4a4 HBASE-18627 Fix TestRegionServerReadRequestMetrics (Chia-Ping Tsai) 2017-08-21 15:38:15 -07:00
Michael Stack 078a6be7d0 HBASE-18634 HBASE-Fix client.TestClientClusterStatus (Chia-Ping Tsai)
This is a revert of a revert; i.e. a reapply just to fix commit message.

This reverts commit fbbae8774c.
2017-08-21 14:38:58 -07:00
Michael Stack fbbae8774c Revert "HBASE-Fix client.TestClientClusterStatus (Chia-Ping Tsai)"
Revert because missing JIRA number

This reverts commit 7b3291fa8b.
2017-08-21 14:38:22 -07:00
Michael Stack 7b3291fa8b HBASE-Fix client.TestClientClusterStatus (Chia-Ping Tsai) 2017-08-21 14:21:30 -07:00
Andrew Purtell 70d48b113e HBASE-18623 Frequent failed to parse at EOF warnings from WALEntryStream 2017-08-21 09:29:24 -07:00
Chia-Ping Tsai 56f9e1a60a HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to next column 2017-08-19 02:15:12 +08:00
Chia-Ping Tsai e9bafeb091 HBASE-18572 Delete can't remove the cells which have no visibility label 2017-08-19 01:55:45 +08:00
tedyu e2532ecd1e HBASE-18617 FuzzyRowKeyFilter should not modify the filter pairs (vinisha) 2017-08-18 09:10:20 -07:00
zhangduo 2af2b4c040 HBASE-18489 Addendum revert protected to private, give shaded import a separated block 2017-08-18 10:54:49 +08:00
Pankaj Kumar 1ab6882f62 HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-17 19:02:39 -07:00
anastas 75a6b36849 HBASE-18375: Fix the bug where the pool chunks from ChunkCreator are deallocated and not returned to pool, because there is no reference to them 2017-08-17 18:23:19 +03:00
Guanghao Zhang 092dc6de84 HBASE-18608 AsyncConnection should return AsyncAdmin interface instead of the implemenation 2017-08-17 09:47:39 +08:00
Michael Stack 473446719b HBASE-18511 Default no regions on master
Changes the configuration hbase.balancer.tablesOnMaster from list of
table names to instead be a boolean; true if master carries
tables/regions and false if it does not.

Adds a new configuration hbase.balancer.tablesOnMaster.systemTablesOnly.
If true, hbase.balancer.tablesOnMaster is considered true but only
system tables are put on the master.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  Master was claiming itself active master though it had stopped. Fix
the activeMaster flag. Set it to false on exit.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
 Add new configs and convenience methods for getting current state of
settings.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
 Move configs up into super Interface and now the settings mean
different, remove the no longer needed processing.
2017-08-16 08:39:36 -07:00
Umesh Agashe acf9b87dca HBASE-18493 [AMv2] Skipped re-assignment of regions on crashed server through AssignmentManager.checkIfShouldMoveSystemRegionAsync() as those regions are handled by ServerCrashProcedure
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-16 08:12:05 -07:00
zhangduo 4c74a73d57 HBASE-18553 Expose scan cursor for asynchronous scanner 2017-08-16 21:04:57 +08:00
Guanghao Zhang 2a9cdd5e75 HBASE-17994 Add async client test to Performance Evaluation tool 2017-08-16 14:05:07 +08:00