Commit Graph

50 Commits

Author SHA1 Message Date
Duo Zhang 8b091c4061
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-26 23:42:37 +08:00
Duo Zhang 9bcb7493e0
HBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master (#4582)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2022-06-29 11:04:21 +08:00
Duo Zhang 02990894ab
HBASE-27121 Set version as 3.0.0-alpha-3 in master in prep for first RC of 3.0.0-alpha-3 (#4535)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2022-06-15 10:51:31 +08:00
Duo Zhang 9c8c9e7fbf HBASE-26899 Run spotless:apply
Closes #4312
2022-05-01 22:15:09 +08:00
Duo Zhang 3f59f21be0
HBASE-26621 Set version as 3.0.0-alpha-3-SNAPSHOT in master (#3978)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-24 14:20:32 +08:00
Duo Zhang 8bca21b47d
HBASE-26558 Set version as 3.0.0-alpha-2 in master in prep for first RC of 3.0.0-alpha-2 (#3935)
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
2021-12-11 20:52:35 +08:00
Duo Zhang 16721239e7
HBASE-26100 Set version as 3.0.0-alpha-2-SNAPSHOT in master (#3508)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-20 23:04:08 +08:00
Duo Zhang 5118321ec9
HBASE-26059 Set version as 3.0.0-alpha-1 in master in prep for first RC of 3.0.0-alpha-1 (#3453)
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-07-02 07:50:41 +08:00
Duo Zhang 302d9ea8b8 HBASE-25373 Remove HTrace completely in code base and try to make use of OpenTelemetry
Signed-off-by: stack <stack@apache.org>
2021-04-25 09:23:23 +08:00
Andrew Purtell 679dd7b8f9
HBASE-25292 Improve InetSocketAddress usage discipline (#2669)
Network identities should be bound late. Remote addresses should be
resolved at the last possible moment, just before connect(). Network
identity mappings can change, so our code should not inappropriately
cache them. Otherwise we might miss a change and fail to operate normally.

Revert "HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors"
Removes hbase.resolve.hostnames.on.failure and related code. We always
resolve hostnames, as late as possible.

Preserve InetSocketAddress caching per RPC connection. Avoids potential
lookups per Call.

Replace InetSocketAddress with Address where used as a map key. If we want
to key by hostname and/or resolved address we should be explicit about it.
Using Address chooses mapping by hostname and port only.

Add metrics for potential nameservice resolution attempts, whenever an
InetSocketAddress is instantiated for connect; and metrics for failed
resolution, whenever InetSocketAddress#isUnresolved on the new instance
is true.

* Use ServerName directly to build a stub key

* Resolve and cache ISA on a RpcChannel as late as possible, at first call

* Remove now invalid unit test TestCIBadHostname

We resolve DNS at the latest possible time, at first call, and do not
resolve hostnames for creating stubs at all, so this unit test cannot
work now.

Reviewed-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-28 21:01:22 +08:00
Duo Zhang 1f66806c96
HBASE-24170 Remove hadoop-2.0 profile (#1495)
Signed-off-by: stack <stack@apache.org>
2020-04-16 18:57:40 +08:00
openinx 5b363a6140 HBASE-22463 Some paths in HFileScannerImpl did not consider block#release which will exhaust the ByteBuffAllocator (#257) 2019-06-24 10:25:33 +08:00
huzheng a8f8a4a1c9 HBASE-22090 The HFileBlock#CacheableDeserializer should pass ByteBuffAllocator to the newly created HFileBlock 2019-06-24 10:25:33 +08:00
Andrew Purtell a30b186568
HBASE-22449 https everywhere in Maven metadata (#247) 2019-05-21 12:34:25 -07:00
Sean Busbey bf140acd20 HBASE-22083 move eclipse settings into a profile.
Signed-off-by: stack <stack@apache.org>
2019-04-25 14:17:18 -05:00
Sean Busbey f1b536bad4 HBASE-20332 shaded mapreduce module shouldn't include hadoop
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports
2018-06-18 11:31:04 -07:00
Zach York d2daada970 HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 17:16:54 -07:00
Chia-Ping Tsai a6eeb26cc0 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:10:23 +08:00
Sean Busbey 2a65066b35 HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2018-03-02 09:25:10 -06:00
zhangduo 7c318cead9 HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules 2018-01-31 20:49:18 +08:00
Apekshit Sharma 0b6422658c HBASE-19780 Remove execution phase and goal of checkstyle plugin.
We don't do checkstyle as part of default "mvn install" since it takes significant
amount of time (~90sec) and slows builds and development. We only need it for pre-commits.
2018-01-23 15:08:04 -08:00
Jan Hentschel 6011641c39 HBASE-19581 Fixed remaining Checkstyle error in hbase-external-blockcache and enabled Checkstyle to fail on violations 2018-01-03 01:32:19 +03:00
Balazs Meszaros f572c4b80e HBASE-10092 Move up on to log4j2
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test

This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.

Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:21:33 -08:00
Mike Drob 75f512bd71 HBASE-18838 Fix hadoop3 check-shaded-invariants 2017-12-15 11:19:47 -06:00
Josh Elser 4a1c3b4210 HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-12-14 15:11:19 -05:00
anoopsamjohn ba4f9f8349 HBASE-19357 Bucket cache no longer L2 for LRU cache. 2017-12-07 15:03:53 +05:30
anoopsamjohn d107e33fcd Revert "HBASE-19357 Bucket cache no longer L2 for LRU cache."
This reverts commit d34e30cad3.
2017-12-06 11:37:09 +05:30
anoopsamjohn d34e30cad3 HBASE-19357 Bucket cache no longer L2 for LRU cache. 2017-12-06 11:25:08 +05:30
Tamas Penzes 377174d3ef HBASE-18601: Update Htrace to 4.2
Updated HTrace version to 4.2
Created TraceUtil class to wrap htrace methods. Uses try with resources.

Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-11 10:34:03 -08:00
Sean Busbey 4b124913f0 HBASE-17823 Migrate to Apache Yetus Audience Annotations
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-12 20:53:30 -05:00
Michael Stack 591d86ab29 HBASE-18782 Module untangling work 2017-09-10 18:10:56 -07:00
Biju Nair f362ef709f HBASE-18737 Display configured max size of memstore and cache on RS UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-09-05 13:01:11 -07:00
Michael Stack fb537fe736 HBASE-18723 [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit
Do a pass with dependency:analyze; remove unused and
explicity list the dependencies we exploit.
Remove the parent dependencies set which had junit, mockito,
log4j, and findbugs annotations (had to put junit back
temporarily in subsequent version of this patch TODO). Listing in
parent set meant these libs were dependencies for all modules
which in practice was not the case. Edited all modules so
those that need any from this parent set now do explicit listing.

Ran the dependency:analyze over the project. Acted on most
suggested removals and requests for explicit listing. Some
grey areas remain around transitives that come in with
hadoop -needs better excludes, another project- and that
the dependency:analyze tool is not always accurate in its
reporting.
2017-08-31 12:41:31 -07:00
Biju Nair 04f114b85c HBASE-18532 Improve cache related stats rendered on RS UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-23 07:53:55 -07:00
Mike Drob 51d458872d HBASE-12349 Add custom error-prone module 2017-08-22 16:38:17 -05:00
Michael Stack 890d92a90c HBASE-17908 Upgrade guava
Pull in guava 22.0 by using the shaded version up in new hbase-thirdparty project.

In poms, exclude guava everywhere except on hadoop-common. Do this so
we minimize transitive includes. hadoop-common is needed because hadoop
Configuration uses guava doing preconditions.

Everywhere we used guava, instead use shaded so fix a load of imports.

Stopwatch API changed as did hashing and toStringHelper which is now
in MoreObjects class. Otherwise, minimal changes to come up on 22.0
2017-07-21 15:28:08 +01:00
Peter Somogyi f2731fc241 HBASE-18264 Update pom plugins
Update plugins in main and subprojects
Unified versions to use variable instead of direct values

Affected plugins:
- apache-rat-plugin 0.11 -> 0.12
- asciidoctor-maven-plugin 1.5.2.1 -> 1.5.5
- asciidoctorj-pdf 1.5.0-alpha.6 -> 1.5.0-alpha.15
- build-helper-maven-plugin 1.9.1 -> 3.0.0
- buildnumber-maven-plugin 1.3 -> 1.4
- exec-maven-plugin 1.2.1/1.4.0 -> 1.6.0
- extra-enforcer-rules 1.0-beta-3 -> 1.0-beta-6
- findbugs-maven-plugin 3.0.0 -> 3.0.4
- jamon-maven-plugin 2.4.1 -> 2.4.2
- maven-bundle-plugin 2.5.3 -> 3.3.0
- maven-compiler-plugin 3.2/3.5.1 -> 3.6.1
- maven-eclipse-plugin 2.9 -> 2.10
- maven-shade-plugin 2.4.1 -> 3.0.0
- maven-surefire-plugin 2.18.1 -> 2.20
- maven-surefire-report-plugin 2.7.2 -> 2.20
- scala-maven-plugin 3.2.0 -> 3.2.2
- spotbugs 3.1.0-RC1 -> 3.1.0-RC3
- wagon-ssh 2.2 -> 2.12
- xml-maven-plugin 1.0 -> 1.0.1

- maven-assembly-plugin 2.4 -> 2.6(inherited)
- maven-dependency-plugin 2.4 -> 2.10 (inherited)
- maven-enforcer-plugin 1.3.1 -> 1.4.1 (inherited)
- maven-javadoc-plugin 2.10.3 -> 2.10.4 (inherited)
- maven-resources-plugin 2.7 (inherited)
- maven-site-plugin 3.4 -> 3.5.1 (inherited)

Change-Id: I84539f555be498dff18caed1e3eea1e1aeb2143a

Signed-off-by: Michael Stack <stack@apache.org>
2017-07-03 19:42:46 -07:00
Michael Stack 929c9dab14 HBASE-18181 Move master branch to version 3.0.0-SNAPSHOT post creation of branch-2 2017-06-06 22:04:39 -07:00
Jan Hentschel b53f354763 HBASE-17532 Replaced explicit type with diamond operator
Signed-off-by: Michael Stack <stack@apache.org>
2017-03-07 11:22:51 -08:00
Michael Stack fb2c89b1b3 HBASE-16785 We are not running all tests
M TestStressWALProcedureStore.java
Disable test that now runs that fails because of difference in pb3.1.0.

Signed-off-by: Michael Stack <stack@apache.org>
2017-01-26 21:49:18 -08:00
Michael Stack 1ee8776273 HBASE-16785 We are not running all tests Do nothing patch just to get baseline of how many tests we run 2017-01-26 12:21:00 -08:00
Jonathan M Hsieh c9d0d11e18 HBASE-12088 Addendum - fix spacing 2016-09-21 20:59:42 -07:00
Jonathan M Hsieh a90d433a2c HBASE-12088 Remove unused hadoop-1.0, hadoop-1.1 profiles from non-root poms 2016-09-21 20:45:03 -07:00
Elliott Clark a71ce6e738 HBASE-14983 Create metrics for per block type hit/miss ratios
Summary: Missing a root index block is worse than missing a data block. We should know the difference

Test Plan: Tested on a local instance. All numbers looked reasonable.

Differential Revision: https://reviews.facebook.net/D55563
2016-03-30 11:41:11 -07:00
stack 12f66e3060 HBASE-15477 Purge 'next block header' from cached blocks
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body.  We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).

This patch undoes this over-persisting.

Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.

There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.

Adds loads of documentation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  (write) Add writing from a ByteBuff.

M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
  (toString) Add one so ByteBuff looks like ByteBuffer when you click on
  it in IDE

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  Remove support for version 1 blocks.

  Cleaned up handling of metadata added when we serialize a block to
  caches. Metadata is smaller now.

  When we serialize (used when caching), do not persist the next blocks
  header if present.

  Removed a bunch of methods, a few of which had overlapping
  functionality and others that exposed too much of our internals.
  Also removed a bunch of constructors and unified the constructors we
  had left over making them share a common init method.
  Shutdown access to defines that should only be used internally here.

  Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
  about metadata saved to caches; was unclear previously what EXTRA was
  about.

  Renamed static final declarations as all uppercase.

  (readBlockDataInternal): Redid. Couldn't make sense of it previously.
  Undid heavy-duty parse of header by constructing HFileBlock. Other
  cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 18:45:17 -07:00
stack 54a543de22 Revert "HBASE-15477 Purge 'next block header' from cached blocks"
Overcommit. Revert to fix.

This reverts commit 000117ad9f.
2016-03-22 18:37:25 -07:00
stack 000117ad9f HBASE-15477 Purge 'next block header' from cached blocks
When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body.  We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting: our block plus the next blocks'
header (33 bytes).

This patch undoes this over-persisting.

Removes support for version 1 blocks (0.2 was added in hbase-0.92.0).
Not needed any more.

There is an open question on whether checksums should be persisted
when caching. The code seems to say no but if cache is SSD backed or
backed by anything that does not do error correction, we'll want
checksums.

Adds loads of documentation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java
  (write) Add writing from a ByteBuff.

M hbase-common/src/main/java/org/apache/hadoop/hbase/nio/ByteBuff.java
  (toString) Add one so ByteBuff looks like ByteBuffer when you click on
  it in IDE

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  Remove support for version 1 blocks.

  Cleaned up handling of metadata added when we serialize a block to
  caches. Metadata is smaller now.

  When we serialize (used when caching), do not persist the next blocks
  header if present.

  Removed a bunch of methods, a few of which had overlapping
  functionality and others that exposed too much of our internals.
  Also removed a bunch of constructors and unified the constructors we
  had left over making them share a common init method.
  Shutdown access to defines that should only be used internally here.

  Renamed all to do w/ 'EXTRA' and 'extraSerialization' to instead talk
  about metadata saved to caches; was unclear previously what EXTRA was
  about.

  Renamed static final declarations as all uppercase.

  (readBlockDataInternal): Redid. Couldn't make sense of it previously.
  Undid heavy-duty parse of header by constructing HFileBlock. Other
  cleanups. Its 1/3rd the length it used to be. More to do in here.
2016-03-22 13:42:38 -07:00
Elliott Clark 7099ab4c56 HBASE-14984 Allow memcached block cache to set optimze to false 2015-12-16 10:35:42 -08:00
zhangduo db42fc64f1 HBASE-14806 Missing sources.jar for several modules when building HBase 2015-11-17 08:54:13 +08:00
Elliott Clark 7b08f4c8be HBASE-14334 Move Memcached block cache in to it's own optional module. 2015-09-16 15:26:38 -07:00