Commit Graph

13068 Commits

Author SHA1 Message Date
Andrew Purtell 8e4516536c HBASE-25292 Improve InetSocketAddress usage discipline (#2669)
Network identities should be bound late. Remote addresses should be
resolved at the last possible moment, just before connect(). Network
identity mappings can change, so our code should not inappropriately
cache them. Otherwise we might miss a change and fail to operate normally.

Revert "HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors"
Removes hbase.resolve.hostnames.on.failure and related code. We always
resolve hostnames, as late as possible.

Preserve InetSocketAddress caching per RPC connection. Avoids potential
lookups per Call.

Replace InetSocketAddress with Address where used as a map key. If we want
to key by hostname and/or resolved address we should be explicit about it.
Using Address chooses mapping by hostname and port only.

Add metrics for potential nameservice resolution attempts, whenever an
InetSocketAddress is instantiated for connect; and metrics for failed
resolution, whenever InetSocketAddress#isUnresolved on the new instance
is true.

* Use ServerName directly to build a stub key

* Resolve and cache ISA on a RpcChannel as late as possible, at first call

* Remove now invalid unit test TestCIBadHostname

We resolve DNS at the latest possible time, at first call, and do not
resolve hostnames for creating stubs at all, so this unit test cannot
work now.

Reviewed-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-04 10:19:53 -08:00
Laxman Goswami 23ef0cb36a
HBASE-25230 Embedded zookeeper server not clean up the old data
Closes #2732

Signed-off-by: maoling <maoling199210191@sina.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-03 14:57:59 +05:30
Akshay Sudheer 298e721839 HBASE-25321 The sort icons not shown after Upgrade JQuery to 3.5.1 (#2694)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-29 21:55:10 +08:00
Lokesh Khurana fac1d4aab7
HBASE-25237 : 'hbase master stop' shuts down the cluster, not the master only (#2705)
Closes #2713

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-27 16:34:37 +05:30
Andrew Purtell 30d988f6e2
HBASE-24640 [branch-1] Purge use of VisibleForTesting (#2697)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-11-25 14:25:24 -08:00
Reid Chan 4640b2224d
HBASE-25285 [branch-1] Fix two methods removed problems reported by compat-check (#2658)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-24 22:17:19 +08:00
Reid Chan edccfe439a
HBASE-25313 [branch-1] Fix the broken pre-commit
Closes #2686

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-21 12:47:41 +05:30
Mate Szalay-Beko a0f55cd439 HBASE-25261 Upgrade Bootstrap to 3.4.1
HBase UI is currently using in bootstrap 3.3.7. This version is vulnerable to 4
medium CVEs (CVE-2018-14040, CVE-2018-14041, CVE-2018-14042, and CVE-2019-8331).
Details on all the bootstrap versions and vulnerabilities is
here: https://snyk.io/vuln/npm:bootstrap

Upgrading to bootstrap 4 would be nice, but potentially more work to do. We
should at least upgrade to the latest bootstrap 3, which is 3.4.1 currently.

closes #2661

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-11-17 14:35:25 +01:00
WenFeiYi 18e9660962
HBASE-25240 gson format of RpcServer.logResponse is abnormal
Closes #2623

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-05 20:25:43 +05:30
Andrew Purtell 6626cc1a33
HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close (#2576)
If hbase.regionserver.close.wait.abort is set to true, interrupt RPC
handler threads holding the region close lock.

Until requests in progress can be aborted, wait on the region close lock for
a configurable interval (specified by hbase.regionserver.close.wait.time.ms,
default 60000 (1 minute)). If we have failed to acquire the close lock after
this interval elapses, if allowed (also specified by
hbase.regionserver.close.wait.abort), abort the regionserver.

We will attempt to interrupt any running handlers every
hbase.regionserver.close.wait.interval.ms (default 10000 (10 seconds)) until
either the close lock is acquired or we reach the maximum wait time.

Define a subset of region operations as interruptible. Track threads holding
the close lock transiting those operations. Set the thread interrupt status
of tracked threads when trying to close the region. Use the thread interrupt
status where safe to break out of request processing.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-03 15:22:47 -08:00
GeorryHuang 16b91c8f49
HBASE-25090 CompactionConfiguration logs unrealistic store file sizes (#2612)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-11-02 13:58:26 -08:00
Nick Dimiduk 41dbba9dce HBASE-25228 Delete dev-support/jenkins_precommit_jira_yetus.sh (#2611)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-11-02 13:21:13 -08:00
Nick Dimiduk e315e63f0c HBASE-24200 Upgrade to Yetus 0.12.0
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-30 14:01:03 -07:00
Sandeep Pal 3238abcfce
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2591)
Patch fixes the single table input format case.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-10-30 12:08:56 -07:00
Andrew Purtell 8813b3bfc1 HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess #2594
Reapply with corrected commit message.

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-29 14:40:21 -07:00
Andrew Purtell 60105e1b22 Revert "HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess (#2594)"
This reverts commit 2e3f842dc7.
2020-10-29 14:40:01 -07:00
Andrew Purtell 2e3f842dc7
HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess (#2594) 2020-10-29 14:33:38 -07:00
sguggilam 0b48208bd5
HBASE-24768 Clear cached service kerberos ticket in case of SASL failures (#2578)
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-10-29 14:23:52 -07:00
Reid Chan b30d1d1180
HBASE-25189 [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level (#2548)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-26 10:46:06 +08:00
Sandeep Pal 9eb3a7918f
HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
Closes #2555

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-23 13:05:55 +05:30
Guanghao Zhang 8e387e9dfa HBASE-25204 Nightly job failed as the name of jdk and maven changed (#2567)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-20 17:32:44 +08:00
Duo Zhang cb10287c8c HBASE-25203 Change the reference url to flaky list in our jenkins jobs (#2566)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-20 17:15:05 +08:00
Duo Zhang 3b2eb3c092 HBASE-25194 Do not publish workspace in flaky find job (#2564)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-20 12:32:35 +08:00
Reid Chan d3ac3420e5
HBASE-25195 [branch-1] getNumOpenConnections is not effective (#2557)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-19 10:54:15 +08:00
WenFeiYi e06695112a
HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll (#2194)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-10-16 20:42:18 +08:00
Viraj Jasani fb25a7d530
HBASE-23935 : Backport HBASE-22978, HBASE-24528, HBASE-24718 to branch-1 (#2312)
* HBASE-22978: Online slow response log (with sub-tasks: Large logs, Replicate logs to HDFS, Filter support in query)
* HBASE-24718: Generic NamedQueue framework for recent in-memory history
* HBASE-24528: Improve balancer decision observability

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-10-16 12:21:17 +05:30
shahrs87 b7d63f96a9
HBASE-25179 : Fix Assert format in HFilePerformanceEvaluation class
Closes #2551

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-16 12:14:12 +05:30
Qi Yu e4a8dcc97c HBASE-25162 Make flaky tests run more aggressively (#2525)
Change the trigger interval from 12h to 4h

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-10 16:11:58 +08:00
Duo Zhang 6f9930532d HBASE-25163 Increase the timeout value for nightly jobs (#2512)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-10 08:33:01 +08:00
Nick Dimiduk 5749747cb2 HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)
Change the test to wait for evidence that the active master has seen
that the backup master killed by the test has gone away. This is done
before proceeding to validate that the dead backup is correctly
omitted from the ClusterStatus report.

Also, minor fixup to several assertions, using `assertEquals` instead
of `assertTrue(...equals(...))` and correcting expected vs. actual
ordering of assertion arguments.

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 14:52:24 -07:00
Duo Zhang 7c103348d0 HBASE-25154 Set java.io.tmpdir to project build directory to avoid writing std*deferred files to /tmp (#2502)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-06 08:43:21 +08:00
Duo Zhang e3c5a7e5fb Revert "HBASE-25134 Migrate HBase PreCommit jenkins job from Hadoop to hbase (#2488)"
This reverts commit b2ba272379.
2020-10-03 21:29:36 +08:00
Duo Zhang b2ba272379 HBASE-25134 Migrate HBase PreCommit jenkins job from Hadoop to hbase (#2488) 2020-10-01 14:51:42 +08:00
Duo Zhang dc89f99108 HBASE-25133 Migrate HBase Nightly jenkins job from Hadoop to hbase (#2487) 2020-10-01 14:46:18 +08:00
Duo Zhang 25b79b7576 HBASE-25132 Migrate flaky test jenkins job from Hadoop to hbase (#2485) 2020-10-01 14:38:52 +08:00
Reid Chan 6edeb5d1ec
HBASE-25122 [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory (#2472)
* Remove unused imports

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-30 18:55:28 +08:00
ankitjain64 e76e7c93d5
HBASE-25119: Fix property name added as part of HBASE-24764
Closes #2471

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-30 10:48:54 +05:30
Reid Chan e719a5b589
HBASE-25114 [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose (#2470)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-30 00:17:40 +08:00
Duo Zhang c7891d802a HBASE-25107 Migrate flaky reporting jenkins job from Hadoop to hbase (#2466)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-29 15:35:51 +08:00
Duo Zhang b3168eebd3 Revert "HBASE-25017 Migrate flaky reporting jenkins job from Hadoop to hbase (#2466)"
This reverts commit bfc6102b1e.
2020-09-29 15:35:43 +08:00
Reid Chan 93b76fdb32
HBASE-25025 [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp (#2396)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-29 14:38:22 +08:00
Duo Zhang bfc6102b1e HBASE-25017 Migrate flaky reporting jenkins job from Hadoop to hbase (#2466)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-28 20:12:26 +08:00
mnpoonia 53d8af5a1a
HBASE-25105 Fix log line in SimpleRegionNormalizer
Closes #2465

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-28 13:11:59 +05:30
Toshihiro Suzuki 53f51f3a7f HBASE-25096 WAL size in RegionServer UI is wrong
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-28 15:09:42 +09:00
Reid Chan ccb4e8907c
HBASE-25030 [Flaky Test] TestRestartCluster#testClusterRestart (#2401)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-28 13:59:28 +08:00
Reid Chan 982aa33cda
HBASE-25031 [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer (#2402)
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2020-09-28 12:23:59 +08:00
Bharath Vissapragada f52d64c4ee
HBASE-25082: Per table WAL metrics: appendCount and appendSize (#2440)
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Ankit Jain <jain.ankit@salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 56c7505f8f)
2020-09-23 21:37:59 -07:00
Andrew Purtell bffc8898c5 HBASE-25079 Upgrade Bootstrap to 3.3.7 (#2442)
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2020-09-23 14:59:40 -07:00
Bharath Vissapragada 55cae10beb
HBASE-23330: Fix delegation token fetch with MasterRegistry
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-19 15:01:06 -07:00
Bharath Vissapragada 3e1450d8b3
HBASE-24765: Dynamic master discovery (#2314)
This patch adds the ability to discover newly added masters
dynamically on the master registry side. The trigger for the
re-fetch is either periodic (5 mins) or any registry RPC failure.
Master server information is cached in masters to avoid repeated
ZK lookups.

Updates the client side connection metrics to maintain a counter
per RPC type so that clients have visibility into counts grouped
by RPC method name.

I didn't add the method to ZK registry interface since there
is a design discussion going on in splittable meta doc. We can
add it later if needed.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 275a38e153)
(cherry picked from commit bb9121da77)
2020-09-19 15:01:06 -07:00