Commit Graph

171 Commits

Author SHA1 Message Date
Guangxu Cheng 3fa3dcd9f9 HBASE-19483 Add proper privilege check for rsgroup commands
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-09 08:11:07 -08:00
Mike Drob 64cb777a8a HBASE-19552 find-and-replace thirdparty offset 2017-12-28 12:01:25 -06:00
Chia-Ping Tsai 97f0aad66e HBASE-19626 Rename Cell.DataType to Cell.Type 2017-12-28 12:59:28 +08:00
Balazs Meszaros 992b5d8630 HBASE-10092 Move up on to log4j2
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test

This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.

Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:58:12 -08:00
Michael Stack 495bedea35
HBASE-19122 Suspect methods on Cell to be deprecated 2017-12-18 15:23:03 -08:00
Chia-Ping Tsai 33d48754f5
HBASE-19462 Deprecate all addImmutable methods in Put
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-14 14:18:04 -08:00
Peter Somogyi 941acc5c05 HBASE-19213 Align check and mutate operations in Table and AsyncTable
- Deprecates old checkAnd*() operations in Table
- Adds Table#CheckAndMutateBuilder and implements it in HTable

Commiter note: When committing the patch, noticed redundant {@inheritDoc} being added in HTable.
Removed new and olds ones.

Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-12-12 20:46:39 -08:00
Jan Hentschel 24b2a5bb1e HBASE-19375 Fixed Checkstyle import order errors in hbase-thrift 2017-12-09 20:19:11 +01:00
Guangxu Cheng 6f7d2afcdd HBASE-19362 Remove unused imports from hbase-thrift module
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2017-11-29 16:32:16 +01:00
Vasudevan 6ac6ae3fa2 HBASE-19092 Make Tag IA.LimitedPrivate and expose for CPs (Ram)
Signed-off-by: Chia-ping Tsai, Anoop Sam John, Stack
2017-11-24 12:38:42 +05:30
Reid Chan e61606231f HBASE-19118 Use SaslUtil to set Sasl.QOP in 'Thrift'
Signed-off-by: Josh Elser <elserj@apache.org>
2017-11-02 00:13:46 -04:00
Michael Stack 250a8bc7ad
HBASE-18995 Move methods that are for internal usage from CellUtil to Private util class (Ramkrishna Vasudevan) 2017-10-27 17:51:32 -07:00
Guanghao Zhang 00f2b18148 HBASE-19007 Align Services Interfaces in Master and RegionServer
Purges Server, MasterServices, and RegionServerServices from
CoprocessorEnvironments. Replaces removed functionality with
a set of carefully curated methods on the *CoprocessorEnvironment
implementations (Varies by CoprocessorEnvironment in that the
MasterCoprocessorEnvironment has Master-type facility exposed,
and so on).

A few core Coprocessors that should long ago have been converted
to be integral, violate their context; e.g. a RegionCoprocessor
wants free access to a hosting RegionServer (which may or may not
be present). Rather than let these violators make us corrupte the
CP API, instead, we've made up a hacky system that allows core
Coprocessors access to internals. A new CoreCoprocessor Annotation
has been introduced. When loading Coprocessors, if the instance is
annotated CoreCoprocessor, we pass it an Environment that has been
padded w/ extra-stuff. On invocation, CoreCoprocessors know how to
route their way to these extras in their environment.

See the *CoprocessoHost for how the do the check for CoreCoprocessor
and pass a fatter *Coprocessor, one that allows getting of either
a RegionServerService or MasterService out of the environment
via Marker Interfaces.

Removed org.apache.hadoop.hbase.regionserver.CoprocessorRegionServerServices

M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
 This Endpoint has been deprecated because its functionality has been
 moved to core. Marking it a CoreCoprocessor in the meantime to
 minimize change.

M hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
 This should be integral to hbase. Meantime, marking it CoreCoprocessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/Server.java
 Added doc on where it is used and added back a few methods we'd
removed.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
 New annotation for core hbase coprocessors. They get richer environment
 on coprocessor loading.

A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasMasterServices.java
A hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/HasRegionServerServices.java
 Marker Interface to access extras if present.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java
  Purge RegionServerServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerCoprocessorEnvironment.java
  Purge MasterServices access. Allow CPs a Connection.

M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterSpaceQuotaObserver.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java
  We no longer have access to MasterServices. Don't need it actually.
  Use short-circuiting Admin instead.

D hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CoprocessorRegionServerServices.java
  Removed. Not needed now we do CP Env differently.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  No need to go via RSS to getOnlineTables; just use HRS.

And so on. Adds tests to ensure we can only get at extra info
if the CP has been properly marked.
2017-10-21 11:07:27 -07:00
Andy Yang 0658252ed6 HBASE-13844 Move static helper methods from KeyValue into CellUtils
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-29 13:20:24 +08:00
Chia-Ping Tsai 6693f45faf HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:41 +08:00
Apekshit Sharma 0c883a23c5 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:45:51 -07:00
anoopsamjohn 0fcc84cadd HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:02:57 +05:30
Guangxu Cheng bb81e9f3ca HBASE-18875 Thrift server supports read-only mode
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-25 15:11:23 -07:00
Sean Busbey d576e5a32d HBASE-17823 Migrate to Apache Yetus Audience Annotations
Includes partial backport of hbase-build-configuration module

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-12 23:15:50 -05:00
Yechao Chen 0501855bc7 HBASE-18772 [JDK8] Replace AtomicLong with LongAdder
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-09-13 06:17:14 +08:00
Michael Stack 7f70b6c8e3 HBASE-18769 Make CompareFilter use generic CompareOperator instead of
internal enum
2017-09-08 10:05:51 -07:00
Peter Somogyi 33711fd481 HBASE-18704 Upgrade hbase to commons-collections 4
Upgrade commons-collections:3.2.2 to commons-collections4:4.1
Add missing dependency for hbase-procedure, hbase-thrift
Replace CircularFifoBuffer with CircularFifoQueue in WALProcedureStore and TaskMonitor

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
(cherry picked from commit 137b105c67)
2017-09-07 10:39:13 -05:00
Umesh Agashe bd219c0fb8 HBASE-18674 upgrading to commons-lang3
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-05 09:46:49 -07:00
Michael Stack 8c84793d22 HBASE-18224 Upgrade Jetty 2017-08-23 23:14:13 -07:00
huzheng 311ca34aec HBASE-18402 Thrift2 should support DeleteFamilyVersion type
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-07-26 15:49:17 -07:00
tedyu b7045d4fcf HBASE-18402 Thrift2 should support DeleteFamilyVersion - revert due to missing JIRA number 2017-07-26 15:49:16 -07:00
huzheng 6abc8bd58e HBASE-18402 Thrift2 should support DeleteFamilyVersion type
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-07-25 17:40:19 -07:00
tedyu edc36b3f90 HBASE-18402 Thrift2 should support DeleteFamilyVersion - revert due to missing JIRA number 2017-07-25 17:40:06 -07:00
huzheng 4e3a750b00 Thrift2 should support DeleteFamilyVersion type
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-07-25 03:10:32 -07:00
Michael Stack d5c6e11016 HBASE-17908 Upgrade guava
Pull in guava 22.0 by using the shaded version up in new hbase-thirdparty project.

In poms, exclude guava everywhere except on hadoop-common. Do this so
we minimize transitive includes. hadoop-common is needed because hadoop
Configuration uses guava doing preconditions.

Everywhere we used guava, instead use shaded so fix a load of imports.

Stopwatch API changed as did hashing and toStringHelper which is now
in MoreObjects class. Otherwise, minimal changes to come up on 22.0
2017-07-21 15:41:52 +01:00
Peter Somogyi 9300fbc9c1 HBASE-16312 update jquery version
Upgrade jquery from 1.8.3 to 3.2.1 in hbase-server and hbase-thrift modules

Change-Id: I92d479e9802d954f607ba409077bc98581e9e5ca

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-20 10:08:12 +01:00
Jan Hentschel 97b649eb93 HBASE-18344 Introduce Append.addColumn as a replacement for Append.add
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-07-13 20:05:22 +08:00
Jan Hentschel 17007685c1 HBASE-13074 Cleaned up usage of hbase.master.lease.thread.wakefrequency
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-05-02 23:12:02 +08:00
lv zehui 9a1aff447e HBASE-17514 emit a warning if thrift1 proxy user is configured but hbase.regionserver.thrift.http is not
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-04-24 11:33:27 -05:00
Michael Stack 0cd4cec5d2 Revert "HBASE-17906 When a huge amount of data writing to hbase through thrift2, there will be a deadlock error. (Albert Lee)"
Mistaken commit.

This reverts commit 9dd5cda017.
2017-04-14 12:07:40 -07:00
Michael Stack 9dd5cda017 HBASE-17906 When a huge amount of data writing to hbase through thrift2, there will be a deadlock error. (Albert Lee) 2017-04-13 21:59:11 -07:00
tedyu 85fda44179 HBASE-17831 Support small scan in thrift2 (Guangxu Cheng) 2017-03-27 10:19:46 -07:00
chenyechao e239e8d2d3 HBASE-17737 Thrift2 proxy should support scan timeRange per column family
--
 .../hadoop/hbase/thrift2/ThriftUtilities.java      |  11 +
 .../hadoop/hbase/thrift2/generated/TScan.java      | 234 ++++++++++++++++++---
 .../org/apache/hadoop/hbase/thrift2/hbase.thrift   |   3 +-
 .../thrift2/TestThriftHBaseServiceHandler.java     |  63 ++++++
 4 files changed, 279 insertions(+), 32 deletions(-)

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-03-08 07:05:44 -08:00
Jan Hentschel b53f354763 HBASE-17532 Replaced explicit type with diamond operator
Signed-off-by: Michael Stack <stack@apache.org>
2017-03-07 11:22:51 -08:00
Jan Hentschel d3bf246421 HBASE-17703 Added random infoport to TestThriftServerCmdLine
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-03-03 12:38:21 -08:00
tedyu eabfa39b27 HBASE-17689 Add support for table.existsAll in thrift2 THBaseservice (Yechao Chen) 2017-02-27 14:40:10 -08:00
tedyu bbc232e4ee HBASE-17689 Add support for table.existsAll in thrift2 THBaseservice - Revert wrong JIRA Id 2017-02-27 14:38:18 -08:00
tedyu e8f9de7851 HBASE-17654 Add support for table.existsAll in thrift2 THBaseservice (Yechao Chen) 2017-02-27 08:32:54 -08:00
Apekshit Sharma f444b3b542 HBASE-17312 Use 'default' keyword in coprocessor Observer interfaces to get rid of 'Base...Observer' implementations. Some javadoc improvements too.
Reason for refactor:
In cases where one might need to use multiple observers, say region, master and regionserver; and the fact that only one class can be extended, it gives rise to following pattern:

public class BaseMasterAndRegionObserver
  extends BaseRegionObserver
  implements MasterObserver

class AccessController
  extends BaseMasterAndRegionObserver
  implements RegionServerObserver

were BaseMasterAndRegionObserver is full copy of BaseMasterObserver.

There is an example of simple case too where the current design fails.
Say only one observer is needed by the coprocessor, but the design doesn't permit extending even that single observer (see RSGroupAdminEndpoint), that leads to copy of full Bas
e...Observer class into coprocessor class leading to 1000s of lines of code and this ugly mix of 5 main functions with 100 useless functions.

Javadocs changes:
- Adds class comments on 'default' methods and expectations.
- Adds explanaiton of Exception handling in Observers' class comment. Removes redundant @throws before each function.
- Improves javadocs for a bunch of functions
- deletes empty @params in a bunch of places

Change-Id: I265738d47e8554e7b4678e88bb916a0cc7d00ab3
2017-02-25 02:49:01 -08:00
Jan Hentschel c7e605445a HBASE-9702 Changed unit tests to use method names for tables
Signed-off-by: Michael Stack <stack@apache.org>
2017-02-22 16:12:22 -08:00
Ashu Pachauri d2c083d21c HBASE-17627 Active workers metric for thrift (Ashu Pachauri)
Signed-off-by: Gary Helmling <garyh@apache.org>
2017-02-15 14:50:19 -08:00
Jan Hentschel 55c2e2d484 HBASE-9702 Changed unit tests to use method names for tables 2017-02-13 13:27:55 -08:00
Gary Helmling 3fe5102506 HBASE-17611 Thrift2 per-call latency metrics are capped at ~2 seconds 2017-02-13 10:16:36 -08:00
Gary Helmling 7a6518f05d HBASE-17578 Thrift metrics should handle exceptions 2017-02-06 11:39:49 -08:00
Michael Stack ffe7dac53d Revert "Revert "HBASE-12894 Upgrade to Jetty 9 for REST / Info Server / Thrift Http Server""
This reverts commit 0ac5d4a717.
This is a revert of a revert; i.e. a reapplication!
Fixes for breakage that comes in with this patch is in a follow-on.
2017-01-30 11:54:54 -08:00