Commit Graph

14337 Commits

Author SHA1 Message Date
huzheng f41250fb37 HBASE-15410 (addendum) Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-26 18:40:05 +08:00
Jan Hentschel c5f47153c4 Added Jan Hentschel to pom.xml 2017-10-26 10:36:08 +02:00
Ramkrishna a899659b16 HBASE-19090 Add config 'hbase.systemtables.compacting.memstore.type' to
hbase-default.xml
2017-10-26 12:42:25 +05:30
Michael Stack fc5dc6282c HBASE-19048 Cleanup MasterObserver hooks which takes IA private params 2017-10-25 23:58:10 -07:00
Ramkrishna 0acfba0e35 HBASE-18994 Decide if META/System tables should use Compacting Memstore or
Default Memstore (Ram)
2017-10-26 10:44:59 +05:30
Peter Somogyi 28d81295f3
HBASE-19029 Align RPC timout methods in Table and AsyncTableBase
As part of HBASE-18978 the rpc timeout methods gets aligned
between Table and AsyncTable interfaces.
Deprecate the following methods in Table:
- int getRpcTimeout()
- int getReadRpcTimeout()
- int getWriteRpcTimeout()
- int getOperationTimeout()

Add the following methods to Table:
- long getRpcTimeout(TimeUnit)
- long getReadRpcTimeout(TimeUnit)
- long getWriteRpcTimeout(TimeUnit)
- long getOperationTimeout(TimeUnit)

Fix some javadoc issues.

Signed-off-by: Michael Stack <stack@apache.org>
2017-10-25 09:54:31 -07:00
Sean Busbey 5255ae6dfa HBASE-19030 nightly runs should attempt to log test results after archiving.
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2017-10-25 10:32:13 -05:00
zhangduo 0b799fdbf0 HBASE-18905 Allow CPs to request flush on Region and know the completion of the requested flush 2017-10-25 20:45:47 +08:00
huzheng 705b3fa98c HBASE-19057 Fix other code review comments about FilterList improvement 2017-10-25 20:36:19 +08:00
huzheng fcaf71d206 HBASE-18368 FilterList with multiple FamilyFilters concatenated by OR does not work
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-10-25 20:36:19 +08:00
huzheng 8a5114b4fa HBASE-18879 HBase FilterList cause KeyOnlyFilter not work 2017-10-25 20:36:19 +08:00
huzheng 59275100e1 HBASE-18411 Dividing FiterList into two separate sub-classes: FilterListWithOR , FilterListWithAND
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-10-25 20:36:19 +08:00
Biju Nair 6442d30f0f HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:36:19 +08:00
huzheng a811fb410d HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-10-25 20:36:19 +08:00
tedyu 10612254a1 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-25 20:36:19 +08:00
huzheng 5716e8d9ed HBASE-17678 FilterList with MUST_PASS_ONE lead to redundancy cells returned - addendum
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:36:19 +08:00
huzheng 45910981b7 HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:36:19 +08:00
Sean Busbey 4e1c7208cb HBASE-18410 disable the HBASE-18957 test until we can fix it on the feature branch. 2017-10-25 20:36:19 +08:00
Michael Stack 43a8ac0015
BASE-19074 Miscellaneous Observer cleanups; ADDEDNUM to fix FindBugs 2017-10-24 22:41:04 -07:00
Michael Stack 95528d12f3
HBASE-19077 Have Region*CoprocessorEnvironment provide an ImmutableOnlineRegions
Change name of Interface OnlineRegions to MutableOnlineRegions.
Change name of Interface ImmutableOnlineRegions to OnlineRegions.
Did this since OnlineRegions is for consumer other than internals.

Add a getOnlineRegions to the RegionCoprocessorEnvironment and to
RegionServerCoprocessorEnvironment so CPs can 'access' local
Regions directly.
2017-10-24 21:58:14 -07:00
Chia-Ping Tsai 2e6481386b HBASE-18754 (addendum) close the input resource 2017-10-25 12:31:37 +08:00
Apekshit Sharma dd70cc3081 HBASE-19073 Cleanup CoordinatedStateManager
- Merged BaseCSM class into CSM interface
- Removed config hbase.coordinated.state.manager.class
- Since state manager is not pluggable anymore, we don't need start/stop/initialize to setup unknown classes. Our internal ZkCSM now requires Server in constructor itself. Makes the dependency clearer too.
- Removed CSM from HRegionServer and HMaster constructor. Although it's a step back from dependency injection, but it's more consistent with our current (not good)  pattern where we initialize everything in the ctor itself.

Change-Id: Ifca06bb354adec5b11ea1bad4707e014410491fc
2017-10-24 19:56:04 -07:00
Misty Stanley-Jones eee3b0180e HBASE-19054 switch precommit image to one from maven
Amending Author: Mike Drob <mdrob@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-10-24 14:51:16 -05:00
Sean Busbey 9353c59a99 HBASE-19018 tests that need bouncycastle must delcare dependency on it.
Signed-off-by: Josh Elser <elserj@apache.org>
2017-10-24 14:30:03 -05:00
tedyu 3cced094c5 HBASE-19065 HRegion#bulkLoadHFiles() should wait for concurrent Region#flush() to finish 2017-10-24 11:53:21 -07:00
Sean Busbey cda2949b81 HBASE-19070 temporarily make the mvnsite nightly test non-voting.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-24 12:06:58 -05:00
Michael Stack cb506fd401 HBASE-19074 Miscellaneous Observer cleanups
Breaks MemStoreSize into MemStoreSize (read-only) and MemStoreSizing
(read/write). MemStoreSize we allow to Coprocesors. MemStoreSizing we
use internally doing MemStore accounting.
2017-10-24 10:02:24 -07:00
Jerry He 9716f62f43 HBASE-19021 Restore a few important missing logics for balancer in 2.0
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-10-24 07:53:17 -07:00
Sean Busbey 1c1906e03a HBASE-19049 Update Kerby to 1.0.1.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-24 09:43:20 -05:00
Chia-Ping Tsai a31e8e34ea HBASE-19066 Correct the directory of openjdk-8 for jenkins
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-10-24 09:18:47 -05:00
Chia-Ping Tsai 93bac3de0a HBASE-18754 Get rid of Writable from TimeRangeTracker 2017-10-24 14:54:34 +08:00
Sreeram Venkatasubramanian 7cdfbde37d HBASE-16290 Dump summary of callQueue content; can help debugging
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-24 14:28:42 +08:00
Apekshit Sharma 3969b853b2 HBASE-19053 Split out o.a.h.h.http from hbase-server into a separate module
Change-Id: Ie3a688b789104df7feaf34ac9fb326a79d6a3960
2017-10-23 22:52:24 -07:00
Michael Stack 456057ef90
HBASE-18846 Accommodate the hbase-indexer/lily/SEP consumer deploy-type
Patch to start a standalone RegionServer that register's itself and
optionally stands up Services. Can work w/o a Master in the mix.
Useful testing. Also can be used by hbase-indexer to put up a
Replication sink that extends public-facing APIs w/o need to extend
internals. See JIRA release note for detail.

This patch adds booleans for whether to start Admin and Client Service.
Other refactoring moves all thread and service start into the one fat
location so we can ask to by-pass 'services' if we don't need them.
See JIRA for an example hbase-server.xml that has config to shutdown
WAL, cache, etc.

Adds checks if a service/thread has been setup before going to use it.

Renames the ExecutorService in HRegionServer from service to
executorService.

See JIRA too for example Connection implementation that makes use of
Connection plugin point to receive a replication stream. The default
replication sink catches the incoming replication stream, undoes the
WALEdits and then creates a Table to call a batch with the
edits; up on JIRA, an example Connection plugin (legit, supported)
returns a Table with an overridden batch method where in we do index
inserts returning appropriate results to keep the replication engine
ticking over.

Upsides: an unadulterated RegionServer that will keep replication metrics
and even hosts a web UI if wanted. No hacks. Just ordained configs
shutting down unused services. Injection of the indexing function at a
blessed point with no pollution by hbase internals; only public imports.
No user of Private nor LimitedPrivate classes.
2017-10-23 21:16:13 -07:00
zhangduo 37b29e909d HBASE-19069 Do not wrap the original CompactionLifeCycleTracker when calling CP hooks 2017-10-24 10:56:14 +08:00
Josh Elser 81133f89fc HBASE-18873 Move protobufs to private implementation on GlobalQuotaSettings
A hack to "hide" the protobufs, but it's not going to be a trivial
change to remove use of protobufs entirely as they're serialized
into the hbase:quota table.
2017-10-23 22:37:10 -04:00
tedyu b7db62c702 HBASE-19072 Missing beak in catch block of InterruptedException in HRegion#waitForFlushes() 2017-10-23 19:34:11 -07:00
Mike Drob a1bc20ab58 HBASE-18893 remove add/delete/modify column 2017-10-23 20:02:25 -05:00
anoopsamjohn 880b26d7d8 HBASE-19067 Do not expose getHDFSBlockDistribution in StoreFile. 2017-10-23 17:04:05 +05:30
zhangduo c9fdbec772 HBASE-18989 Polish the compaction related CP hooks 2017-10-23 16:44:54 +08:00
anoopsamjohn 4add40ca24 HBASE-19046 RegionObserver#postCompactSelection Avoid passing shaded ImmutableList param. 2017-10-23 12:14:09 +05:30
huzheng 24931044d6 Add Zheng Hu to pom.xml 2017-10-23 13:41:45 +08:00
Xiang Li 2ee8690b47 HBASE-18824 Add meaningful comment to HConstants.LATEST_TIMESTAMP to explain why it is MAX_VALUE
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-22 04:47:00 +08:00
Guanghao Zhang 38879fb3ff HBASE-19007 Align Services Interfaces in Master and RegionServer
Purges Server, MasterServices, and RegionServerServices from
CoprocessorEnvironments. Replaces removed functionality with
a set of carefully curated methods on the *CoprocessorEnvironment
implementations (Varies by CoprocessorEnvironment in that the
MasterCoprocessorEnvironment has Master-type facility exposed,
and so on).

A few core Coprocessors that should long ago have been converted
to be integral, violate their context; e.g. a RegionCoprocessor
wants free access to a hosting RegionServer (which may or may not
be present). Rather than let these violators make us corrupte the
CP API, instead, we've made up a hacky system that allows core
Coprocessors access to internals. A new CoreCoprocessor Annotation
has been introduced. When loading Coprocessors, if the instance is
annotated CoreCoprocessor, we pass it an Environment that has been
padded w/ extra-stuff. On invocation, CoreCoprocessors know how to
route their way to these extras in their environment.

See the *CoprocessoHost for how the do the check for CoreCoprocessor
and pass a fatter *Coprocessor, one that allows getting of either
a RegionServerService or MasterService out of the environment
via Marker Interfaces.

Removed org.apache.hadoop.hbase.regionserver.CoprocessorRegionServerServices

M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
 This Endpoint has been deprecated because its functionality has been
 moved to core. Marking it a CoreCoprocessor in the meantime to
 minimize change.

M hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
 This should be integral to hbase. Meantime, marking it CoreCoprocessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/Server.java
 Added doc on where it is used and added back a few methods we'd
removed.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
 New annotation for core hbase coprocessors. They get richer environment
 on coprocessor loading.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasMasterServices.java
A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasRegionServerServices.java
 Marker Interface to access extras if present.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java
  Purge RegionServerServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterSpaceQuotaObserver.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java
  We no longer have access to MasterServices. Don't need it actually.
  Use short-circuiting Admin instead.

D hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorRegionServerServices.java
  Removed. Not needed now we do CP Env differently.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  No need to go via RSS to getOnlineTables; just use HRS.

And so on. Adds tests to ensure we can only get at extra info
if the CP has been properly marked.
2017-10-21 11:06:30 -07:00
Guanghao Zhang 592d541f5d HBASE-19010 Reimplement getMasterInfoPort for Admin 2017-10-21 18:19:22 +08:00
Chia-Ping Tsai cb5c4776de HBASE-19058. The wget isn't installed in building docker image
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-10-20 20:35:33 -05:00
Sean Busbey b10ad9e97f HBASE-19039 refactor shadedjars test to only run on java changes.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-20 19:35:20 -05:00
Sean Busbey dd4dbae764 HBASE-19060 precommit plugin test 'hadoopcheck' should only run when java or maven files change.
Signed-off-by: Mike Drob <mdrob@apache.org>
2017-10-20 19:35:14 -05:00
Mike Drob 89d3b0b07f HBASE-19061 update enforcer rules for NPE 2017-10-20 16:04:16 -05:00
anoopsamjohn 64d164b86d HBASE-19045 Deprecate RegionObserver#postInstantiateDeleteTracker. 2017-10-20 23:57:40 +05:30