Commit Graph

1731 Commits

Author SHA1 Message Date
zhangduo fad7d01d8f HBASE-18972 Use Builder pattern to remove nullable parameters for coprocessor methods in RawAsyncTable interface 2017-11-02 13:58:29 +08:00
Apekshit Sharma 71a55dcd64 HBASE-18925 Update mockito dependency from mockito-all:1.10.19 to mockito-core:2.1.0 for JDK8 support.
Last mockito-all release was in Dec'14. Mockito-core has had many releases since then.

From mockito's site:
- "Mockito does not produce the mockito-all artifact anymore ; this one was primarily
aimed at ant users, and contained other dependencies. We felt it was time to move on
and remove such artifacts as they cause problems in dependency management system like
maven or gradle."
- anyX() and any(SomeType.class) matchers now reject nulls and check type.
2017-11-01 14:21:38 -07:00
eshcar 17e7aff37e HBASE-16417: In-memory MemStore Policy for Flattening and Compactions 2017-11-01 09:26:00 +02:00
Michael Stack 8237fdbd1b HBASE-18770 Remove bypass method in ObserverContext and implement the
'bypass' logic case by case

Changes Coprocessor ObserverContext 'bypass' semantic. We flip the
default so bypass is NOT supported on Observer invocations; only a
couple of preXXX methods in RegionObserver allow it: e.g.  preGet
and prePut but not preFlush, etc. Everywhere else, we throw
a DoesNotSupportBypassException if a Coprocessor Observer
tries to invoke bypass. Master Observers can no longer stop
or change move, split, assign, create table, etc.

Ditto on complete, the mechanism that allowed a Coprocessor
rule that all subsequent Coprocessors are skipped in an
invocation chain; now, complete is only available to
bypassable methods (and Coprocessors will get an exception if
they try to 'complete' when it is not allowed).

See javadoc for whether a Coprocessor Observer method supports
'bypass'. If no mention, 'bypass' is NOT supported.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 Added passing of 'bypassable' (and 'completable') and default 'result' argument to
 the Operation constructors rather than pass the excecution engine as parameters.
 Makes it so can clean up RegionObserverHost and make the calling
 clearer regards what is going on.
 Methods that support 'bypass' must set this flag on the Observer.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Refactoring in here is minor. A few methods that used support bypass
 no longer do so removed the check and the need of an if/else meant a
 left-shift in some code.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 Ditto

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
 In here label explicitly those methods that are bypassable.
 Some changes to make sure we call the corresponding execOperation.

TestMasterObserver had a bunch of test of bypass method. All removed or
disabled.

TODO: What to do w/ the Scanner methods.
2017-10-31 09:52:20 -07:00
Peter Somogyi 482d6bd3a4 HBASE-19031 Align exist method in Table and AsyncTable interfaces
Deprecate Table::existsAll method and add Table::exists.
RemoteHTable already had a deprecated exists method, remove that
and implement the new exists from Table interface.

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-29 07:11:27 +08:00
Michael Stack f6c2490b82
HBASE-18995 Move methods that are for internal usage from CellUtil to Private util class (Ramkrishna Vasudevan) 2017-10-27 17:48:55 -07:00
Tamas Penzes 82b1c320f0 HBASE-13346: Clean up Filter package for post 1.0 s/KeyValue/Cell/g
Added filterCell method to Filter, it calls filterKeyValue by default
Deprecated filterKeyValue in Filter, bud added default functionality to return Filter.ReturnCode.INCLUDE.
Added filterKeyValue (calling filterCell) to Filters extending FilterBase to be backward compatible.
renamed filterKeyValue to filterCell in all implementations
changed all internal calls to use filterCell instead of filterKeyValue
changed tests too

This way the change is simple and backward compatible.
Any existing custom filter should work since they override filterKeyValue
and the implementation is called by Filter.filterCell.

Moved FilterWrapper to hbase-server

Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2017-10-27 11:16:13 +05:30
QilinCao 459202bab0 HBASE-19091 Code annotation wrote BinaryComparator instead of LongComparator
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2017-10-26 21:37:28 +02:00
huzheng f41250fb37 HBASE-15410 (addendum) Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-26 18:40:05 +08:00
Peter Somogyi 28d81295f3
HBASE-19029 Align RPC timout methods in Table and AsyncTableBase
As part of HBASE-18978 the rpc timeout methods gets aligned
between Table and AsyncTable interfaces.
Deprecate the following methods in Table:
- int getRpcTimeout()
- int getReadRpcTimeout()
- int getWriteRpcTimeout()
- int getOperationTimeout()

Add the following methods to Table:
- long getRpcTimeout(TimeUnit)
- long getReadRpcTimeout(TimeUnit)
- long getWriteRpcTimeout(TimeUnit)
- long getOperationTimeout(TimeUnit)

Fix some javadoc issues.

Signed-off-by: Michael Stack <stack@apache.org>
2017-10-25 09:54:31 -07:00
huzheng 705b3fa98c HBASE-19057 Fix other code review comments about FilterList improvement 2017-10-25 20:36:19 +08:00
huzheng fcaf71d206 HBASE-18368 FilterList with multiple FamilyFilters concatenated by OR does not work
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-10-25 20:36:19 +08:00
huzheng 8a5114b4fa HBASE-18879 HBase FilterList cause KeyOnlyFilter not work 2017-10-25 20:36:19 +08:00
huzheng 59275100e1 HBASE-18411 Dividing FiterList into two separate sub-classes: FilterListWithOR , FilterListWithAND
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-10-25 20:36:19 +08:00
Biju Nair 6442d30f0f HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:36:19 +08:00
huzheng a811fb410d HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-10-25 20:36:19 +08:00
tedyu 10612254a1 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-25 20:36:19 +08:00
huzheng 5716e8d9ed HBASE-17678 FilterList with MUST_PASS_ONE lead to redundancy cells returned - addendum
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:36:19 +08:00
huzheng 45910981b7 HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-10-25 20:36:19 +08:00
Josh Elser 81133f89fc HBASE-18873 Move protobufs to private implementation on GlobalQuotaSettings
A hack to "hide" the protobufs, but it's not going to be a trivial
change to remove use of protobufs entirely as they're serialized
into the hbase:quota table.
2017-10-23 22:37:10 -04:00
Guanghao Zhang 38879fb3ff HBASE-19007 Align Services Interfaces in Master and RegionServer
Purges Server, MasterServices, and RegionServerServices from
CoprocessorEnvironments. Replaces removed functionality with
a set of carefully curated methods on the *CoprocessorEnvironment
implementations (Varies by CoprocessorEnvironment in that the
MasterCoprocessorEnvironment has Master-type facility exposed,
and so on).

A few core Coprocessors that should long ago have been converted
to be integral, violate their context; e.g. a RegionCoprocessor
wants free access to a hosting RegionServer (which may or may not
be present). Rather than let these violators make us corrupte the
CP API, instead, we've made up a hacky system that allows core
Coprocessors access to internals. A new CoreCoprocessor Annotation
has been introduced. When loading Coprocessors, if the instance is
annotated CoreCoprocessor, we pass it an Environment that has been
padded w/ extra-stuff. On invocation, CoreCoprocessors know how to
route their way to these extras in their environment.

See the *CoprocessoHost for how the do the check for CoreCoprocessor
and pass a fatter *Coprocessor, one that allows getting of either
a RegionServerService or MasterService out of the environment
via Marker Interfaces.

Removed org.apache.hadoop.hbase.regionserver.CoprocessorRegionServerServices

M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
 This Endpoint has been deprecated because its functionality has been
 moved to core. Marking it a CoreCoprocessor in the meantime to
 minimize change.

M hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
 This should be integral to hbase. Meantime, marking it CoreCoprocessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/Server.java
 Added doc on where it is used and added back a few methods we'd
removed.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
 New annotation for core hbase coprocessors. They get richer environment
 on coprocessor loading.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasMasterServices.java
A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasRegionServerServices.java
 Marker Interface to access extras if present.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java
  Purge RegionServerServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterSpaceQuotaObserver.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java
  We no longer have access to MasterServices. Don't need it actually.
  Use short-circuiting Admin instead.

D hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorRegionServerServices.java
  Removed. Not needed now we do CP Env differently.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  No need to go via RSS to getOnlineTables; just use HRS.

And so on. Adds tests to ensure we can only get at extra info
if the CP has been properly marked.
2017-10-21 11:06:30 -07:00
Guanghao Zhang 592d541f5d HBASE-19010 Reimplement getMasterInfoPort for Admin 2017-10-21 18:19:22 +08:00
Michael Stack d798541261 HBASE-19043 Purge TableWrapper and CoprocessorHConnnection
Also purge Coprocessor#getTable... Let Coprocessors manage their
Table Connections in hbase2.0.0.
2017-10-20 11:06:10 -07:00
Mike Drob 5facaded90 HBASE-16338 Remove Jackson1 deps
* Change imports from org.codehaus to com.fasterxml
* Exclude transitive jackson1 from hadoop and others
* Minor test cleanup to add assert messages, fix some parameter order
* Add anti-pattern check for using jackson 1 imports
* Add explicit non-null serialization directive to ScannerModel
2017-10-20 09:20:12 -05:00
Jerry He a43a00e89c HBASE-10367 RegionServer graceful stop / decommissioning
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-10-19 21:54:45 -07:00
Ramkrishna 70f4c5da47 HBSE-18945 Make a IA.LimitedPrivate interface for CellComparator (Ram) 2017-10-17 23:17:07 +05:30
Reid Chan 9f61f8b281 HBASE-18990 ServerLoad doesn't override #equals which leads to #equals in ClusterStatus always false
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-18 00:37:13 +08:00
Guanghao Zhang 51489b2081 HBASE-18914 Remove AsyncAdmin's methods which were already deprecated in Admin interface 2017-10-16 22:16:04 +08:00
Chia-Ping Tsai 240b4b16ff HBASE-18997 Remove the redundant methods in RegionInfo 2017-10-14 23:26:11 +08:00
tedyu ffdc0e6851 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-10-13 13:21:20 -07:00
anoopsamjohn b4ed130083 HBASE-18183 Region interface cleanup for CP expose. 2017-10-11 16:47:15 +05:30
Mike Drob 690ff38f54 HBASE-18981 Address issues found by error-prone in hbase-client 2017-10-10 14:07:54 -05:00
zhangduo 8597b19b3d HBASE-18951 Use Builder pattern to remove nullable parameters for checkAndXXX methods in RawAsyncTable/AsyncTable interface 2017-10-10 14:41:23 +08:00
Sean Busbey b727ab850c Revert "HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned"
This reverts commit 0d0c330401.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:49:20 -05:00
Sean Busbey f97c0bd8b5 Revert "HBASE-17678 FilterList with MUST_PASS_ONE lead to redundancy cells returned - addendum"
This reverts commit 0eeafa04f2.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:49:20 -05:00
Sean Busbey 4eea0d923e Revert "HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT"
This reverts commit df34300cd3.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:49:20 -05:00
Sean Busbey 183b3e31bd Revert "HBASE-18160 Fix incorrect logic in FilterList.filterKeyValue"
This reverts commit f54cc1ca51.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:49:20 -05:00
Sean Busbey e8fa9cc85f Revert "HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()"
This reverts commit 3bd824faca.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 16:49:20 -05:00
Maytee Chinavanichkit 08274045c9 HBASE-18921 Fix Result.current ArrayIndexOutOfBoundsException
Patch ArrayIndexOutOfBoundsException when current() is called after
advance() has already returned false

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-10-08 02:50:51 +08:00
Guanghao Zhang 63440a9c7d HBASE-18909 Deprecate Admin's methods which used String regex 2017-10-07 21:16:01 +08:00
Chia-Ping Tsai 11aa6742f0 HBASE-18927 Add the DataType which is subset of KeyValue#Type to CellBuilder for building cell 2017-10-04 23:45:43 +08:00
Ramkrishna 50265395d1 HBASE-18490 Modifying a table descriptor to enable replicas does not
create replica regions (Ram)
2017-10-03 13:10:44 +05:30
Chia-Ping Tsai d35d8376a7 HBASE-18897 Substitute MemStore for Memstore 2017-10-02 20:55:06 +08:00
Apekshit Sharma cacf3f5835 HBASE-18884 (addendum) Add more javadoc comment. Coprocessor Design Improvements follow up of HBASE-17732.
Change-Id: Id1a9e508cc04612b1e79e6cfa7e39b7755598be3
2017-09-29 17:32:11 -07:00
Robert Yokota c835dcc7e7 HBASE-18559 Add histogram to MetricsConnection to track concurrent calls per server
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-09-29 17:12:26 -07:00
Andrew Purtell ca87d05a51 HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 2017-09-29 15:37:04 -07:00
Biju Nair 3bd824faca HBASE-18904 Missing break in NEXT_ROW case of FilterList#mergeReturnCodeForOrOperator()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-29 15:31:10 -07:00
Andy Yang afce850cfd HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 11:50:23 +08:00
Michael Stack b5b4108fce HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732; MINOR AMENDMENT adding README to design-doc dir 2017-09-28 13:24:40 -07:00
Apekshit Sharma 74d0adce61 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:25:47 -07:00