13062 Commits

Author SHA1 Message Date
Reid Chan
edccfe439a
HBASE-25313 [branch-1] Fix the broken pre-commit
Closes #2686

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-21 12:47:41 +05:30
Mate Szalay-Beko
a0f55cd439 HBASE-25261 Upgrade Bootstrap to 3.4.1
HBase UI is currently using in bootstrap 3.3.7. This version is vulnerable to 4
medium CVEs (CVE-2018-14040, CVE-2018-14041, CVE-2018-14042, and CVE-2019-8331).
Details on all the bootstrap versions and vulnerabilities is
here: https://snyk.io/vuln/npm:bootstrap

Upgrading to bootstrap 4 would be nice, but potentially more work to do. We
should at least upgrade to the latest bootstrap 3, which is 3.4.1 currently.

closes #2661

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-11-17 14:35:25 +01:00
WenFeiYi
18e9660962
HBASE-25240 gson format of RpcServer.logResponse is abnormal
Closes #2623

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-05 20:25:43 +05:30
Andrew Purtell
6626cc1a33
HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close (#2576)
If hbase.regionserver.close.wait.abort is set to true, interrupt RPC
handler threads holding the region close lock.

Until requests in progress can be aborted, wait on the region close lock for
a configurable interval (specified by hbase.regionserver.close.wait.time.ms,
default 60000 (1 minute)). If we have failed to acquire the close lock after
this interval elapses, if allowed (also specified by
hbase.regionserver.close.wait.abort), abort the regionserver.

We will attempt to interrupt any running handlers every
hbase.regionserver.close.wait.interval.ms (default 10000 (10 seconds)) until
either the close lock is acquired or we reach the maximum wait time.

Define a subset of region operations as interruptible. Track threads holding
the close lock transiting those operations. Set the thread interrupt status
of tracked threads when trying to close the region. Use the thread interrupt
status where safe to break out of request processing.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-03 15:22:47 -08:00
GeorryHuang
16b91c8f49
HBASE-25090 CompactionConfiguration logs unrealistic store file sizes (#2612)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-11-02 13:58:26 -08:00
Nick Dimiduk
41dbba9dce HBASE-25228 Delete dev-support/jenkins_precommit_jira_yetus.sh (#2611)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-11-02 13:21:13 -08:00
Nick Dimiduk
e315e63f0c HBASE-24200 Upgrade to Yetus 0.12.0
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-30 14:01:03 -07:00
Sandeep Pal
3238abcfce
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2591)
Patch fixes the single table input format case.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-10-30 12:08:56 -07:00
Andrew Purtell
8813b3bfc1 HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess #2594
Reapply with corrected commit message.

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-29 14:40:21 -07:00
Andrew Purtell
60105e1b22 Revert "HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess (#2594)"
This reverts commit 2e3f842dc7b5961c3cd3a84d76dd50575a0ff76d.
2020-10-29 14:40:01 -07:00
Andrew Purtell
2e3f842dc7
HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess (#2594) 2020-10-29 14:33:38 -07:00
sguggilam
0b48208bd5
HBASE-24768 Clear cached service kerberos ticket in case of SASL failures (#2578)
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-10-29 14:23:52 -07:00
Reid Chan
b30d1d1180
HBASE-25189 [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level (#2548)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-26 10:46:06 +08:00
Sandeep Pal
9eb3a7918f
HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
Closes #2555

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-23 13:05:55 +05:30
Guanghao Zhang
8e387e9dfa HBASE-25204 Nightly job failed as the name of jdk and maven changed (#2567)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-20 17:32:44 +08:00
Duo Zhang
cb10287c8c HBASE-25203 Change the reference url to flaky list in our jenkins jobs (#2566)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-20 17:15:05 +08:00
Duo Zhang
3b2eb3c092 HBASE-25194 Do not publish workspace in flaky find job (#2564)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-20 12:32:35 +08:00
Reid Chan
d3ac3420e5
HBASE-25195 [branch-1] getNumOpenConnections is not effective (#2557)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-19 10:54:15 +08:00
WenFeiYi
e06695112a
HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll (#2194)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-10-16 20:42:18 +08:00
Viraj Jasani
fb25a7d530
HBASE-23935 : Backport HBASE-22978, HBASE-24528, HBASE-24718 to branch-1 (#2312)
* HBASE-22978: Online slow response log (with sub-tasks: Large logs, Replicate logs to HDFS, Filter support in query)
* HBASE-24718: Generic NamedQueue framework for recent in-memory history
* HBASE-24528: Improve balancer decision observability

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-10-16 12:21:17 +05:30
shahrs87
b7d63f96a9
HBASE-25179 : Fix Assert format in HFilePerformanceEvaluation class
Closes #2551

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-16 12:14:12 +05:30
Qi Yu
e4a8dcc97c HBASE-25162 Make flaky tests run more aggressively (#2525)
Change the trigger interval from 12h to 4h

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-10 16:11:58 +08:00
Duo Zhang
6f9930532d HBASE-25163 Increase the timeout value for nightly jobs (#2512)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-10 08:33:01 +08:00
Nick Dimiduk
5749747cb2 HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)
Change the test to wait for evidence that the active master has seen
that the backup master killed by the test has gone away. This is done
before proceeding to validate that the dead backup is correctly
omitted from the ClusterStatus report.

Also, minor fixup to several assertions, using `assertEquals` instead
of `assertTrue(...equals(...))` and correcting expected vs. actual
ordering of assertion arguments.

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 14:52:24 -07:00
Duo Zhang
7c103348d0 HBASE-25154 Set java.io.tmpdir to project build directory to avoid writing std*deferred files to /tmp (#2502)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-06 08:43:21 +08:00
Duo Zhang
e3c5a7e5fb Revert "HBASE-25134 Migrate HBase PreCommit jenkins job from Hadoop to hbase (#2488)"
This reverts commit b2ba2723792fd3ed4116ab5fed8458209749b4b3.
2020-10-03 21:29:36 +08:00
Duo Zhang
b2ba272379 HBASE-25134 Migrate HBase PreCommit jenkins job from Hadoop to hbase (#2488) 2020-10-01 14:51:42 +08:00
Duo Zhang
dc89f99108 HBASE-25133 Migrate HBase Nightly jenkins job from Hadoop to hbase (#2487) 2020-10-01 14:46:18 +08:00
Duo Zhang
25b79b7576 HBASE-25132 Migrate flaky test jenkins job from Hadoop to hbase (#2485) 2020-10-01 14:38:52 +08:00
Reid Chan
6edeb5d1ec
HBASE-25122 [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory (#2472)
* Remove unused imports

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-30 18:55:28 +08:00
ankitjain64
e76e7c93d5
HBASE-25119: Fix property name added as part of HBASE-24764
Closes #2471

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-30 10:48:54 +05:30
Reid Chan
e719a5b589
HBASE-25114 [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose (#2470)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-30 00:17:40 +08:00
Duo Zhang
c7891d802a HBASE-25107 Migrate flaky reporting jenkins job from Hadoop to hbase (#2466)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-29 15:35:51 +08:00
Duo Zhang
b3168eebd3 Revert "HBASE-25017 Migrate flaky reporting jenkins job from Hadoop to hbase (#2466)"
This reverts commit bfc6102b1e575db20a932a0a6738331b04c3f1f8.
2020-09-29 15:35:43 +08:00
Reid Chan
93b76fdb32
HBASE-25025 [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp (#2396)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-29 14:38:22 +08:00
Duo Zhang
bfc6102b1e HBASE-25017 Migrate flaky reporting jenkins job from Hadoop to hbase (#2466)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-28 20:12:26 +08:00
mnpoonia
53d8af5a1a
HBASE-25105 Fix log line in SimpleRegionNormalizer
Closes #2465

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-28 13:11:59 +05:30
Toshihiro Suzuki
53f51f3a7f HBASE-25096 WAL size in RegionServer UI is wrong
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-28 15:09:42 +09:00
Reid Chan
ccb4e8907c
HBASE-25030 [Flaky Test] TestRestartCluster#testClusterRestart (#2401)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-28 13:59:28 +08:00
Reid Chan
982aa33cda
HBASE-25031 [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer (#2402)
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2020-09-28 12:23:59 +08:00
Bharath Vissapragada
f52d64c4ee
HBASE-25082: Per table WAL metrics: appendCount and appendSize (#2440)
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Ankit Jain <jain.ankit@salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 56c7505f8f3c59261693bd8631cfcc4e709ead23)
2020-09-23 21:37:59 -07:00
Andrew Purtell
bffc8898c5 HBASE-25079 Upgrade Bootstrap to 3.3.7 (#2442)
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2020-09-23 14:59:40 -07:00
Bharath Vissapragada
55cae10beb
HBASE-23330: Fix delegation token fetch with MasterRegistry
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-19 15:01:06 -07:00
Bharath Vissapragada
3e1450d8b3
HBASE-24765: Dynamic master discovery (#2314)
This patch adds the ability to discover newly added masters
dynamically on the master registry side. The trigger for the
re-fetch is either periodic (5 mins) or any registry RPC failure.
Master server information is cached in masters to avoid repeated
ZK lookups.

Updates the client side connection metrics to maintain a counter
per RPC type so that clients have visibility into counts grouped
by RPC method name.

I didn't add the method to ZK registry interface since there
is a design discussion going on in splittable meta doc. We can
add it later if needed.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 275a38e1533eafa1d4bd1d50c13bcecd9a397ea8)
(cherry picked from commit bb9121da77c7b881a3cc4c389029a610fc2b0925)
2020-09-19 15:01:06 -07:00
Bharath Vissapragada
ebe9e68274
HBASE-23305: Implement master based registry for client connections
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-19 15:01:06 -07:00
Bharath Vissapragada
d866994635
HBASE-23604: Clarify Registry usage in the code
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-19 15:01:06 -07:00
stack
cb756629b0
HBASE-7767 Get rid of ZKTable, and table enable/disable state in ZK (Andrey Stepachev)
Signed-off-by: Andrew Purtell <apurtell@apache.org>

(cherry picked from commit 3cc5d19039904361f60c413f10f3cbca27a7ba96)
2020-09-19 15:01:06 -07:00
Bharath Vissapragada
18200b09e2
HBASE-23304: RPCs needed for client meta information lookup (#904)
* HBASE-23304: RPCs needed for client meta information lookup

This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.

Fixed a bunch of checkstyle nits around the places the patch
touches.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 4f8fbba0c01742f17fa2d85a4b944d7f42b7c2b1)
(cherry picked from commit 488460e84015647840afca8d02ab279653b11dee)
2020-09-19 15:01:05 -07:00
Bharath Vissapragada
9a1d5a02b0
HBASE-23281: Track meta region locations in masters (#830)
* HBASE-23281: Track meta region changes on masters

This patch adds a simple cache that tracks the meta region replica
locations. It keeps an eye on the region movements so that the
cached locations are not stale.

This information is used for servicing client RPCs for connections
that use master based registry (HBASE-18095). The RPC end points
will be added in a separate patch.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8571d389cfe7bb18dafad82ca011e78390a21061)
(cherry picked from commit 89581d9d218436dcaa840f895b2e98c5d4b65a1e)
2020-09-19 15:01:05 -07:00
Bharath Vissapragada
e4161e5875
HBASE-23275: Track active master's address in ActiveMasterManager (#812)
Currently we just track whether an active master exists.
It helps to also track the address of the active master in
all the masters to help serve the client RPC requests to
know which master is active.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit efebb843afe4458599e12cf3390fe534780fac4e)
(cherry picked from commit 742949165ff63a9bab448e4d1e40a386aa715c82)
2020-09-19 15:01:05 -07:00