Commit Graph

161 Commits

Author SHA1 Message Date
Mike Drob 75f512bd71 HBASE-18838 Fix hadoop3 check-shaded-invariants 2017-12-15 11:19:47 -06:00
Chia-Ping Tsai 70f02dbc7c
HBASE-19462 Deprecate all addImmutable methods in Put
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-14 14:15:54 -08:00
Josh Elser 4a1c3b4210 HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-12-14 15:11:19 -05:00
zhangduo 54827cf613 HBASE-19251 Merge RawAsyncTable and AsyncTable 2017-11-16 14:36:28 +08:00
Tamas Penzes 377174d3ef HBASE-18601: Update Htrace to 4.2
Updated HTrace version to 4.2
Created TraceUtil class to wrap htrace methods. Uses try with resources.

Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-11 10:34:03 -08:00
Josh Elser 0bdfd4f449 HBASE-19002 Add some more RegionObserver examples
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-09 11:14:35 -05:00
zhangduo 28cdf4afb8 HBASE-19095 Add CP hooks in RegionObserver for in memory compaction 2017-11-05 20:28:05 +08:00
Apekshit Sharma 71a55dcd64 HBASE-18925 Update mockito dependency from mockito-all:1.10.19 to mockito-core:2.1.0 for JDK8 support.
Last mockito-all release was in Dec'14. Mockito-core has had many releases since then.

From mockito's site:
- "Mockito does not produce the mockito-all artifact anymore ; this one was primarily
aimed at ant users, and contained other dependencies. We felt it was time to move on
and remove such artifacts as they cause problems in dependency management system like
maven or gradle."
- anyX() and any(SomeType.class) matchers now reject nulls and check type.
2017-11-01 14:21:38 -07:00
zhangduo e0a530e714 HBASE-19033 Allow CP users to change versions and TTL before opening StoreScanner 2017-10-29 21:53:11 +08:00
zhangduo 0b799fdbf0 HBASE-18905 Allow CPs to request flush on Region and know the completion of the requested flush 2017-10-25 20:45:47 +08:00
zhangduo d653c0bc7f HBASE-18747 Introduce new example and helper classes to tell CP users how to do filtering on scanners 2017-10-14 08:48:09 +08:00
anoopsamjohn b4ed130083 HBASE-18183 Region interface cleanup for CP expose. 2017-10-11 16:47:15 +05:30
Chia-Ping Tsai 11aa6742f0 HBASE-18927 Add the DataType which is subset of KeyValue#Type to CellBuilder for building cell 2017-10-04 23:45:43 +08:00
Peter Somogyi 0af61dce65
HBASE-18815 We need to pass something like CompactionRequest in CP to give user some information about the compaction
CompactionRequest was removed from CP in HBASE-18453, this change reintroduces
CompatcionRequest to CP as a read-only interface called CompactionRequest.
The CompactionRequest class is renamed to CompactionRequestImpl.

Additionally, this change removes selectionTimeInNanos from CompactionRequest and
uses selectionTime as a replacement. This means that CompactionRequest:toString
is modified and compare as well.

Signed-off-by: Michael Stack <stack@apache.org>
2017-10-03 07:52:33 -07:00
Apekshit Sharma 74d0adce61 HBASE-18884 Coprocessor Design Improvements follow up of HBASE-17732
- Change Service Coprocessor#getService() to List<Service> Coprocessor#getServices()
- Checkin the finalized design doc into repo
- Added example to javadoc of Coprocessor base interface on how to implement one in the new design
2017-09-28 10:25:47 -07:00
Chia-Ping Tsai a11a35a113 HBASE-18839 Apply RegionInfo to code base 2017-09-28 20:19:29 +08:00
zhangduo 7f4c3b3564 HBASE-18826 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface 2017-09-28 15:26:52 +08:00
Apekshit Sharma 97513466c0 HBASE-17732 Coprocessor Design Improvements
------------------------------------------------------
TL;DR
------------------------------------------------------
We are moving from Inheritence
- Observer *is* Coprocessor
- FooService *is* CoprocessorService
To Composition
- Coprocessor *has* Observer
- Coprocessor *has* Service

------------------------------------------------------
Design Changes
------------------------------------------------------
- Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
  WALCoprocessor
- These new *Coprocessor interfaces have a get*Observer() function for each observer type
  supported by them.
- Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
  get it from the base interface.
- Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
  own trickery.
- CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
  Deleted if can, else marked @VisibleForTesting.

------------------------------------------------------
Backward Compatibility
------------------------------------------------------
- Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
- Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
- Old coprocessors implementing CoprocessorService (for master/region host)
  /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
- Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
- Note that if a coprocessor implements both observer and service in same class, its service
  component will continue to work but it's observer component won't work.

------------------------------------------------------
Notes
------------------------------------------------------
Did a side-by-side comparison of CPs in master and after patch. These coprocessors which were just
CoprocessorService earlier, needed a home in some coprocessor in new design. For most it was clear
since they were using a particular type of environment. Some were tricky.

- JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense for
  processes?)
- RSGroupAdminEndpoint --> MasterCP
- VisibilityController -> MasterCP and RegionCP

These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
which can only come from a RegionCPHost.
- AggregateImplementation
- BaseRowProcessorEndpoint
- BulkDeleteEndpoint
- Export
- RefreshHFilesEndpoint
- RowCountEndpoint
- MultiRowMutationEndpoint
- SecureBulkLoadEndpoint
- TokenProvider

Change-Id: I813145f2bc11815f52ac703563b879962c249764
2017-09-27 12:40:25 -07:00
anoopsamjohn bd68551f26 HBASE-18298 RegionServerServices Interface cleanup for CP expose. 2017-09-27 11:01:51 +05:30
zhangduo a5f84430a3 HBASE-18825 Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface 2017-09-25 09:35:39 +08:00
Chia-Ping Tsai a6c3c645fd HBASE-18823 Apply RegionInfo to MasterObserver/RegionObserver/WALObserver 2017-09-20 18:43:11 -07:00
zhangduo 61d10feffa HBASE-18453 CompactionRequest should not be exposed to user directly 2017-09-14 20:37:33 +08:00
zhangduo e6e52cd80f HBASE-18793 Remove deprecated methods in RegionObserver 2017-09-13 11:33:03 +08:00
Michael Stack 591d86ab29 HBASE-18782 Module untangling work 2017-09-10 18:10:56 -07:00
Michael Stack fb537fe736 HBASE-18723 [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit
Do a pass with dependency:analyze; remove unused and
explicity list the dependencies we exploit.
Remove the parent dependencies set which had junit, mockito,
log4j, and findbugs annotations (had to put junit back
temporarily in subsequent version of this patch TODO). Listing in
parent set meant these libs were dependencies for all modules
which in practice was not the case. Edited all modules so
those that need any from this parent set now do explicit listing.

Ran the dependency:analyze over the project. Acted on most
suggested removals and requests for explicit listing. Some
grey areas remain around transitives that come in with
hadoop -needs better excludes, another project- and that
the dependency:analyze tool is not always accurate in its
reporting.
2017-08-31 12:41:31 -07:00
Chia-Ping Tsai 7465973068 HBASE-15806 An endpoint-based export tool 2017-08-30 14:06:04 +08:00
Chia-Ping Tsai d63a287bfd HBASE-18519 Use builder pattern to create cell 2017-08-28 14:14:49 +08:00
Apekshit Sharma 664b6be0ef HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
  on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
  These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
  This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
  confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
  more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
  big win!

Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-25 18:38:48 -07:00
zhangduo 8d33949b8d HBASE-16324 Remove LegacyScanQueryMatcher 2017-08-26 08:04:43 +08:00
Ajay Jadhav 98bb5c05e3 HBASE-18448 Added refresh HFiles coprocessor endpoint
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2017-08-25 06:56:41 +05:30
anoopsamjohn 88356029f1 Revert "HBASE-18448 Added refresh HFiles coprocessor endpoint"
This reverts commit 612c23556d.
2017-08-24 21:13:08 +05:30
Ajay Jadhav 612c23556d HBASE-18448 Added refresh HFiles coprocessor endpoint
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2017-08-24 20:42:49 +05:30
Mike Drob 51d458872d HBASE-12349 Add custom error-prone module 2017-08-22 16:38:17 -05:00
Michael Stack c6ac04ab39 HBASE-18271 Shade netty Purge mention of netty-all.
Add in hbase-thirdparty hbase-shaded-netty instead.
s/io.netty/org.apache.hadoop.hbase.shaded.io.netty/ everywhere in hbase.

Also set a system property when running tests and when starting
hbase; required by netty so can find the relocation files in the
bundled .so.
2017-08-13 12:46:21 -07:00
Tamas Penzes 8da77b4146 HBASE-18387: [Thrift] Make principal configurable in DemoClient.java
Added optional (fourth) parameter "server-principal"
The solution is backward compatible, in case not given, uses "hbase" as default value
If the third parameter is skipped the fourth cannot be set.

Signed-off-by: Josh Elser <elserj@apache.org>
2017-08-10 23:47:07 -04:00
Chia-Ping Tsai fd76eb39d7 HBASE-18502 Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor 2017-08-07 11:26:15 +08:00
Michael Stack 7a6de1bd42 HBASE-17056 Remove checked in PB generated files
Selective add of dependency on hbase-thirdparty jars.
Update to READMEs on how protobuf is done (and update to refguide).
Removed all checked in generated protobuf files. They are generated
on the fly now as part of mainline build.
2017-08-02 09:33:20 -07:00
Michael Stack 890d92a90c HBASE-17908 Upgrade guava
Pull in guava 22.0 by using the shaded version up in new hbase-thirdparty project.

In poms, exclude guava everywhere except on hadoop-common. Do this so
we minimize transitive includes. hadoop-common is needed because hadoop
Configuration uses guava doing preconditions.

Everywhere we used guava, instead use shaded so fix a load of imports.

Stopwatch API changed as did hashing and toStringHelper which is now
in MoreObjects class. Otherwise, minimal changes to come up on 22.0
2017-07-21 15:28:08 +01:00
Chia-Ping Tsai bc8ebc6f72 HBASE-18241 Change client.Table, client.Admin, Region, Store, and HBaseTestingUtility to not use HTableDescriptor or HColumnDescriptor 2017-07-08 16:54:25 +08:00
Michael Stack 6786b2b63e Revert "HBASE-17056 Remove checked in PB generated files Selective add of dependency on"
Revert for now. Build unstable and some interesting issues around
CLASSPATH

This reverts commit df93c13fd2.
2017-07-06 21:58:32 -07:00
Michael Stack df93c13fd2 HBASE-17056 Remove checked in PB generated files Selective add of dependency on
hbase-thirdparty jars. Update to READMEs on how protobuf is done (and update to
refguide) Removed all checked in generated protobuf files. They are generatedon
the fly now as part of mainline build.
2017-07-05 20:57:11 -07:00
Chia-Ping Tsai 8b63eb6fc6 HBASE-18314 Eliminate the findbugs warnings for hbase-examples 2017-07-05 09:54:40 +08:00
Peter Somogyi f2731fc241 HBASE-18264 Update pom plugins
Update plugins in main and subprojects
Unified versions to use variable instead of direct values

Affected plugins:
- apache-rat-plugin 0.11 -> 0.12
- asciidoctor-maven-plugin 1.5.2.1 -> 1.5.5
- asciidoctorj-pdf 1.5.0-alpha.6 -> 1.5.0-alpha.15
- build-helper-maven-plugin 1.9.1 -> 3.0.0
- buildnumber-maven-plugin 1.3 -> 1.4
- exec-maven-plugin 1.2.1/1.4.0 -> 1.6.0
- extra-enforcer-rules 1.0-beta-3 -> 1.0-beta-6
- findbugs-maven-plugin 3.0.0 -> 3.0.4
- jamon-maven-plugin 2.4.1 -> 2.4.2
- maven-bundle-plugin 2.5.3 -> 3.3.0
- maven-compiler-plugin 3.2/3.5.1 -> 3.6.1
- maven-eclipse-plugin 2.9 -> 2.10
- maven-shade-plugin 2.4.1 -> 3.0.0
- maven-surefire-plugin 2.18.1 -> 2.20
- maven-surefire-report-plugin 2.7.2 -> 2.20
- scala-maven-plugin 3.2.0 -> 3.2.2
- spotbugs 3.1.0-RC1 -> 3.1.0-RC3
- wagon-ssh 2.2 -> 2.12
- xml-maven-plugin 1.0 -> 1.0.1

- maven-assembly-plugin 2.4 -> 2.6(inherited)
- maven-dependency-plugin 2.4 -> 2.10 (inherited)
- maven-enforcer-plugin 1.3.1 -> 1.4.1 (inherited)
- maven-javadoc-plugin 2.10.3 -> 2.10.4 (inherited)
- maven-resources-plugin 2.7 (inherited)
- maven-site-plugin 3.4 -> 3.5.1 (inherited)

Change-Id: I84539f555be498dff18caed1e3eea1e1aeb2143a

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-03 19:42:46 -07:00
Peter Somogyi 61534931cc HBASE-18291 Regenerate thrift2 python examples
Change thrift python example to version 0.9.3
Update instructions in DemoClient.py

Change-Id: I5d2c2698c7f1c848b2c8bfaa761a6e1376811be3

Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-07-03 09:25:28 -07:00
zhangduo cb88b64629 HBASE-17008 Examples to make AsyncClient go down easy 2017-06-14 10:24:19 +08:00
Michael Stack 929c9dab14 HBASE-18181 Move master branch to version 3.0.0-SNAPSHOT post creation of branch-2 2017-06-06 22:04:39 -07:00
Xiang Li 1367519cd0 HBASE-17947 Location of Examples.proto is wrong in comment of RowCountEndPoint.java
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-04-25 01:48:57 -07:00
eshcar 8f4ae0a0dc HBASE-17655 Removing MemStoreScanner and SnapshotScanner 2017-03-21 12:35:47 +02:00
Jan Hentschel b53f354763 HBASE-17532 Replaced explicit type with diamond operator
Signed-off-by: Michael Stack <stack@apache.org>
2017-03-07 11:22:51 -08:00
Andrew Purtell 404a2883f2 HBASE-17722 Metrics subsystem stop/start messages add a lot of useless bulk to operational logging 2017-03-03 12:40:06 -08:00