Commit Graph

29 Commits

Author SHA1 Message Date
Jan Hentschel 6011641c39 HBASE-19581 Fixed remaining Checkstyle error in hbase-external-blockcache and enabled Checkstyle to fail on violations 2018-01-03 01:32:19 +03:00
Balazs Meszaros f572c4b80e HBASE-10092 Move up on to log4j2
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test

This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.

Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:21:33 -08:00
Mike Drob 75f512bd71 HBASE-18838 Fix hadoop3 check-shaded-invariants 2017-12-15 11:19:47 -06:00
Josh Elser 4a1c3b4210 HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-12-14 15:11:19 -05:00
anoopsamjohn ba4f9f8349 HBASE-19357 Bucket cache no longer L2 for LRU cache. 2017-12-07 15:03:53 +05:30
anoopsamjohn d107e33fcd Revert "HBASE-19357 Bucket cache no longer L2 for LRU cache."
This reverts commit d34e30cad3.
2017-12-06 11:37:09 +05:30
anoopsamjohn d34e30cad3 HBASE-19357 Bucket cache no longer L2 for LRU cache. 2017-12-06 11:25:08 +05:30
Tamas Penzes 377174d3ef HBASE-18601: Update Htrace to 4.2
Updated HTrace version to 4.2
Created TraceUtil class to wrap htrace methods. Uses try with resources.

Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-11 10:34:03 -08:00
Sean Busbey 4b124913f0 HBASE-17823 Migrate to Apache Yetus Audience Annotations
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-12 20:53:30 -05:00
Michael Stack 591d86ab29 HBASE-18782 Module untangling work 2017-09-10 18:10:56 -07:00
Biju Nair f362ef709f HBASE-18737 Display configured max size of memstore and cache on RS UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-05 13:01:11 -07:00
Michael Stack fb537fe736 HBASE-18723 [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit
Do a pass with dependency:analyze; remove unused and
explicity list the dependencies we exploit.
Remove the parent dependencies set which had junit, mockito,
log4j, and findbugs annotations (had to put junit back
temporarily in subsequent version of this patch TODO). Listing in
parent set meant these libs were dependencies for all modules
which in practice was not the case. Edited all modules so
those that need any from this parent set now do explicit listing.

Ran the dependency:analyze over the project. Acted on most
suggested removals and requests for explicit listing. Some
grey areas remain around transitives that come in with
hadoop -needs better excludes, another project- and that
the dependency:analyze tool is not always accurate in its
reporting.
2017-08-31 12:41:31 -07:00
Biju Nair 04f114b85c HBASE-18532 Improve cache related stats rendered on RS UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-23 07:53:55 -07:00
Mike Drob 51d458872d HBASE-12349 Add custom error-prone module 2017-08-22 16:38:17 -05:00
Michael Stack 890d92a90c HBASE-17908 Upgrade guava
Pull in guava 22.0 by using the shaded version up in new hbase-thirdparty project.

In poms, exclude guava everywhere except on hadoop-common. Do this so
we minimize transitive includes. hadoop-common is needed because hadoop
Configuration uses guava doing preconditions.

Everywhere we used guava, instead use shaded so fix a load of imports.

Stopwatch API changed as did hashing and toStringHelper which is now
in MoreObjects class. Otherwise, minimal changes to come up on 22.0
2017-07-21 15:28:08 +01:00
Peter Somogyi f2731fc241 HBASE-18264 Update pom plugins
Update plugins in main and subprojects
Unified versions to use variable instead of direct values

Affected plugins:
- apache-rat-plugin 0.11 -> 0.12
- asciidoctor-maven-plugin 1.5.2.1 -> 1.5.5
- asciidoctorj-pdf 1.5.0-alpha.6 -> 1.5.0-alpha.15
- build-helper-maven-plugin 1.9.1 -> 3.0.0
- buildnumber-maven-plugin 1.3 -> 1.4
- exec-maven-plugin 1.2.1/1.4.0 -> 1.6.0
- extra-enforcer-rules 1.0-beta-3 -> 1.0-beta-6
- findbugs-maven-plugin 3.0.0 -> 3.0.4
- jamon-maven-plugin 2.4.1 -> 2.4.2
- maven-bundle-plugin 2.5.3 -> 3.3.0
- maven-compiler-plugin 3.2/3.5.1 -> 3.6.1
- maven-eclipse-plugin 2.9 -> 2.10
- maven-shade-plugin 2.4.1 -> 3.0.0
- maven-surefire-plugin 2.18.1 -> 2.20
- maven-surefire-report-plugin 2.7.2 -> 2.20
- scala-maven-plugin 3.2.0 -> 3.2.2
- spotbugs 3.1.0-RC1 -> 3.1.0-RC3
- wagon-ssh 2.2 -> 2.12
- xml-maven-plugin 1.0 -> 1.0.1

- maven-assembly-plugin 2.4 -> 2.6(inherited)
- maven-dependency-plugin 2.4 -> 2.10 (inherited)
- maven-enforcer-plugin 1.3.1 -> 1.4.1 (inherited)
- maven-javadoc-plugin 2.10.3 -> 2.10.4 (inherited)
- maven-resources-plugin 2.7 (inherited)
- maven-site-plugin 3.4 -> 3.5.1 (inherited)

Change-Id: I84539f555be498dff18caed1e3eea1e1aeb2143a

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-03 19:42:46 -07:00
Michael Stack 929c9dab14 HBASE-18181 Move master branch to version 3.0.0-SNAPSHOT post creation of branch-2 2017-06-06 22:04:39 -07:00
Jan Hentschel b53f354763 HBASE-17532 Replaced explicit type with diamond operator
Signed-off-by: Michael Stack <stack@apache.org>
2017-03-07 11:22:51 -08:00
Michael Stack fb2c89b1b3 HBASE-16785 We are not running all tests
M TestStressWALProcedureStore.java
Disable test that now runs that fails because of difference in pb3.1.0.

Signed-off-by: Michael Stack <stack@apache.org>
2017-01-26 21:49:18 -08:00
Michael Stack 1ee8776273 HBASE-16785 We are not running all tests Do nothing patch just to get baseline of how many tests we run 2017-01-26 12:21:00 -08:00
Jonathan M Hsieh c9d0d11e18 HBASE-12088 Addendum - fix spacing 2016-09-21 20:59:42 -07:00
Jonathan M Hsieh a90d433a2c HBASE-12088 Remove unused hadoop-1.0, hadoop-1.1 profiles from non-root poms 2016-09-21 20:45:03 -07:00
Elliott Clark a71ce6e738 HBASE-14983 Create metrics for per block type hit/miss ratios
Summary: Missing a root index block is worse than missing a data block. We should know the difference

Test Plan: Tested on a local instance. All numbers looked reasonable.

Differential Revision: https://reviews.facebook.net/D55563
2016-03-30 11:41:11 -07:00
stack 12f66e3060 HBASE-15477 Purge 'next block header' from cached blocks
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body.  We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).

This patch undoes this over-persisting.

Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.

There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.

Adds loads of documentation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  (write) Add writing from a ByteBuff.

M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
  (toString) Add one so ByteBuff looks like ByteBuffer when you click on
  it in IDE

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  Remove support for version 1 blocks.

  Cleaned up handling of metadata added when we serialize a block to
  caches. Metadata is smaller now.

  When we serialize (used when caching), do not persist the next blocks
  header if present.

  Removed a bunch of methods, a few of which had overlapping
  functionality and others that exposed too much of our internals.
  Also removed a bunch of constructors and unified the constructors we
  had left over making them share a common init method.
  Shutdown access to defines that should only be used internally here.

  Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
  about metadata saved to caches; was unclear previously what EXTRA was
  about.

  Renamed static final declarations as all uppercase.

  (readBlockDataInternal): Redid. Couldn't make sense of it previously.
  Undid heavy-duty parse of header by constructing HFileBlock. Other
  cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 18:45:17 -07:00
stack 54a543de22 Revert "HBASE-15477 Purge 'next block header' from cached blocks"
Overcommit. Revert to fix.

This reverts commit 000117ad9f.
2016-03-22 18:37:25 -07:00
stack 000117ad9f HBASE-15477 Purge 'next block header' from cached blocks
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body.  We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).

This patch undoes this over-persisting.

Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.

There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.

Adds loads of documentation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  (write) Add writing from a ByteBuff.

M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
  (toString) Add one so ByteBuff looks like ByteBuffer when you click on
  it in IDE

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  Remove support for version 1 blocks.

  Cleaned up handling of metadata added when we serialize a block to
  caches. Metadata is smaller now.

  When we serialize (used when caching), do not persist the next blocks
  header if present.

  Removed a bunch of methods, a few of which had overlapping
  functionality and others that exposed too much of our internals.
  Also removed a bunch of constructors and unified the constructors we
  had left over making them share a common init method.
  Shutdown access to defines that should only be used internally here.

  Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
  about metadata saved to caches; was unclear previously what EXTRA was
  about.

  Renamed static final declarations as all uppercase.

  (readBlockDataInternal): Redid. Couldn't make sense of it previously.
  Undid heavy-duty parse of header by constructing HFileBlock. Other
  cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 13:42:38 -07:00
Elliott Clark 7099ab4c56 HBASE-14984 Allow memcached block cache to set optimze to false 2015-12-16 10:35:42 -08:00
zhangduo db42fc64f1 HBASE-14806 Missing sources.jar for several modules when building HBase 2015-11-17 08:54:13 +08:00
Elliott Clark 7b08f4c8be HBASE-14334 Move Memcached block cache in to it's own optional module. 2015-09-16 15:26:38 -07:00