Commit Graph

15308 Commits

Author SHA1 Message Date
Todd Lipcon 3673bfc241 HBASE-20403. Fix race between prefetch task and non-pread HFile reads
With prefetch-on-open enabled, the task doing the prefetching was using
non-positional (i.e. streaming) reads. If the main (non-prefetch) thread
was also using non-positional reads, these two would conflict, because
inputstreams are not thread-safe for non-positional reads.

In the case of an encrypted filesystem, this could cause JVM crashes,
etc, as underlying cipher buffers were freed underneath the racing
threads. In the case of a non-encrypted filesystem, less severe errors
would be thrown. The included unit test reproduces the latter case.

(cherry picked from commit 025ddce868)
Signed-off-by: Todd Lipcon <todd@cloudera.com>
2018-06-25 12:12:30 -07:00
Michael Stack d6cea08efe
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do
General log cleanup; setting stuff that can flood the log to TRACE.
2018-06-25 12:12:03 -07:00
Guangxu Cheng 5f104c21eb HBASE-19764 Fix Checkstyle errors in hbase-endpoint
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2018-06-24 10:59:37 +02:00
Michael Stack 7c45f02110 HBASE-20778 Make it so WALPE runs on DFS 2018-06-23 23:34:40 -07:00
Michael Stack ca88874995 HBASE-20710 extra cloneFamily() in Mutation.add(Cell); ADDENDUM checkstyle 2018-06-23 20:42:06 -07:00
Huaxiang Sun 633bab0c7b HBASE-20710 extra cloneFamily() in Mutation.add(Cell)
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-23 20:37:57 -07:00
zhangduo eb67404cef HBASE-20775 TestMultiParallel is flakey 2018-06-24 08:42:53 +08:00
zhangduo f3061a67fc HBASE-18569 Add prefetch support for async region locator 2018-06-22 18:20:21 +08:00
zhangduo a86141b625 HBASE-20752 Make sure the regions are truly reopened after ReopenTableRegionsProcedure 2018-06-22 14:06:29 +08:00
zhangduo 6cebe06225 HBASE-20767 Always close hbaseAdmin along with connection in HBTU 2018-06-22 10:20:06 +08:00
Ankit Singhal 28d0d8c5cc HBASE-20642 Clients should re-use the same nonce across DDL operations
Also changes modify table operations to help the case where a MTP spans
two master, avoiding the sanity-checks propagating back to the client
unnecessarily.

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-20 15:10:52 -07:00
Josh Elser c8b76eb3f1 HBASE-20706 Prevent MTP from trying to reopen non-OPEN regions
ModifyTableProcedure is using MoveRegionProcedure in a way
that was unintended from the original implementation. As such,
we have to guard against certain usages of it. We know we can
re-open OPEN regions, but regions in OPENING will similarly
soon be OPEN (thus, we want to reopen those regions too).

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-20 14:30:36 -07:00
zhangduo 5c2cb15e0b HBASE-20739 Add priority for SCP 2018-06-20 15:18:47 +08:00
taiynlee 8aabe36a49 HBASE-20737 (addendum) put collection into ArrayList instead of addAll function -- RetriesExhaustedWithDetailsException.java
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-20 14:46:04 +08:00
zhangduo 83969b0da1 HBASE-20742 Always create WAL directory for region server 2018-06-20 14:21:28 +08:00
Michael Stack 9eeb501825 HBASE-20745 Log when master proc wal rolls 2018-06-19 19:53:29 -07:00
zhangduo 3e33aecea2 HBASE-20708 Remove the usage of RecoverMetaProcedure in master startup 2018-06-19 15:09:11 +08:00
Sean Busbey 0a4ae9fa94 HBASE-20334 add a test that verifies basic client and MR integration
Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-18 14:03:51 -07:00
Sean Busbey 133892aae2 HBASE-20615 emphasize shaded artifacts in client tarball.
Conflicts:
	bin/hbase

Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-18 14:03:28 -07:00
Josh Elser 1725094e6b HBASE-19735 Create a client-tarball assembly
Provides an extra client descriptor to build a second
tarball with a reduced set of dependencies. Not of great
impact now, but will build the way for better in the future.

Signed-off-by: Sean Busbey <busbey@apache.org>

 Conflicts:
	hbase-assembly/pom.xml

 Conflicts:
	hbase-spark/pom.xml
2018-06-18 14:03:22 -07:00
Sean Busbey e19fdd3903 HBASE-20333 Provide a shaded client that allows downstream to provide Hadoop needs.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-18 14:03:13 -07:00
Sean Busbey ee84a8f243 HBASE-20332 shaded mapreduce module shouldn't include hadoop
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports

 Conflicts:
	hbase-backup/pom.xml
	hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml

Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-18 14:02:48 -07:00
Mike Drob b04c976fe6 HBASE-20478 Update checkstyle to v8.2
Cannot go to latest (8.9) yet due to
  https://github.com/checkstyle/checkstyle/issues/5279

* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors

Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-18 14:02:40 -07:00
taiynlee 8edd5d948a HBASE-20737 put collection into ArrayList instead of addAll function
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-17 11:16:16 +08:00
tedyu b2afba580b HBASE-20723 Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them 2018-06-16 01:34:53 -07:00
Josh Elser 4e9f5e8d64 HBASE-20681 Explicitly include hamcrest in binary tarball
HBase ITs require junit which requires hamcrest. Hadoop recently
stopped including hamcrest in their installation (and thus our inherited
classpath), which means that we need to ship it.

Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-15 11:12:57 -05:00
Xu Cang b68746c0b2 HBASE-20695 Implement table level RegionServer replication metrics
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-06-15 10:45:13 +08:00
Sean Busbey a4cb98b1cd HBASE-20733 QABot should run checkstyle tests if the checkstyle configs change
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-14 20:26:42 -05:00
jingyuntian bde9f08a83 HBASE-20625 refactor some WALCellCodec related code
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-06-14 19:46:33 +08:00
zhangduo 161dc7c7f3 HBASE-20722 Make RegionServerTracker only depend on children changed event 2018-06-14 08:38:53 +08:00
Guanghao Zhang 075523dd1e HBASE-20561 The way we stop a ReplicationSource may cause the RS down 2018-06-13 18:05:27 +08:00
Mike Drob 0a6aec4981 HBASE-19377 Update Java API CC version
Compatibility checker complaining about hash collisions, newer versions
use longer id strings.
2018-06-12 15:13:35 -05:00
Rajeshbabu Chintaguntla c0d67209df HBASE-20635 Add missing shaded-pb conversion method for UserPermission
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-06-11 15:13:17 -04:00
Kevin Risden a8ea49bfb7 HBASE-19852 HBase Thrift should use a SPNEGO HTTP/hostname principal for checking HTTP Kerberos authentication
Return 401 sooner when AUTHORIZATION header is missing

HBase Thrift server was checking for the AUTHORIZATION header and assuming it was always present
even when it was the first request. Many clients will not send the AUTHORIZATION header until
a 401 is received. HBase Thrift in the case of no header was throwing multiple exceptions and
filling the logs with exceptions. This was fixed by checking that if the AUTHORIZATION header is
empty then return a 401 immediately if security is enabled.

Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-11 13:37:29 -04:00
Balazs Meszaros d44e8a7aff HBASE-20656 Validate pre-2.0 coprocessors against HBase 2.0+
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-11 10:32:40 -05:00
Mike Drob 4b0bbd839e HBASE-20707 Move MissingSwitchDefault case check
Perform this check using error-prone instead of checkstyle because the
former can handle enum switches somewhat more intelligently.
2018-06-11 10:13:29 -05:00
zhangduo 6befdc43ba HBASE-20700 Move meta region when server crash can cause the procedure to be stuck 2018-06-11 15:28:21 +08:00
Guanghao Zhang 4d971d0f48 HBASE-20698 (addendum) Master don't record right server version until new started region server call regionServerReport method 2018-06-10 08:32:01 +08:00
Guanghao Zhang 9d15e16946 HBASE-20698 Master don't record right server version until new started region server call regionServerReport method 2018-06-09 14:47:07 +08:00
Nihal Jain 4a5fe54d94 HBASE-20699 QuotaCache should cancel the QuotaRefresherChore service inside its stop()
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-06-08 07:22:52 -07:00
Michael Stack 858eee20ec HBASE-20702 Processing crash, skip ONLINE'ing empty rows
Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-07 09:54:32 -07:00
eric-maynard 271d93dc73 HBASE-20665: Changed log level of HBASE-8547 warning to debug
Closes #77

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-07 11:34:52 -04:00
Peter Somogyi 42ca976803 HBASE-20689 Docker fails to install rubocop for precommit
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-06 23:51:47 +02:00
Peter Somogyi 00289b8ffa HBASE-20683 Incorrect return value for PreUpgradeValidator
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-06 20:04:22 +02:00
EugeneNik 323eccc65b HBASE-20684 org.apache.hadoop.hbase.client.Scan#setStopRow javadoc uses incorrect method
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-05 21:58:06 +08:00
Guanghao Zhang 0b0257b412 HBASE-20678 NPE in ReplicationSourceManager#NodeFailoverWorker 2018-06-05 14:59:22 +08:00
Andrew Purtell d7b09de854 HBASE-20670 NPE in HMaster#isInMaintenanceMode 2018-06-04 15:19:45 -07:00
Nihal Jain 3673e54a40 HBASE-20677 Backport test of HBASE-20566 'Creating a system table after enabling rsgroup feature puts region into RIT' to branch-2
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-06-04 13:24:51 -07:00
Michael Stack 063eefe3b0 HBASE-20634 Reopen region while server crash can cause the procedure to be stuck; ADDENDUM 2018-06-04 12:38:56 -07:00
Michael Stack 27e2c8c86b HBASE-20628 SegmentScanner does over-comparing when one flushing
Signed-off-by: eshcar <eshcar@oath.com>
Signed-off-by: anoopsjohn <anoopsamjohn@gmail.com>
2018-06-04 09:50:13 -07:00