Commit Graph

1560 Commits

Author SHA1 Message Date
Duo Zhang 39d143f290
HBASE-26020 Split TestWALEntryStream.testDifferentCounts out (#3409)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-06-23 22:46:07 +08:00
Wei-Chiu Chuang cb247f9464 HBASE-26019 Remove the reflection and call Configuration.getPassword() directly. (#3408)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Wellington Chevreuil <wchevreuil@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
2021-06-23 13:24:31 +02:00
Peter Somogyi 1a9ddb6694 Revert "Remove the reflection and call Configuration.getPassword() directly. (#3408)"
This reverts commit 9a324bd4d0.
2021-06-23 13:24:08 +02:00
Wei-Chiu Chuang 9a324bd4d0
Remove the reflection and call Configuration.getPassword() directly. (#3408)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Wellington Chevreuil <wchevreuil@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
2021-06-22 18:15:02 -07:00
Andrew Purtell 97f90e0be2
HBASE-25994 Active WAL tailing fails when WAL value compression is enabled (#3377)
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.

Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-14 17:16:31 -07:00
Andrew Purtell 335305e0cf
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302)
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-06-01 09:57:48 -07:00
Michael Stack f119a865cf
HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO (#3334)
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-29 09:02:52 -07:00
Baiqiang Zhao 479ae88be3
HBASE-25928 TestHBaseConfiguration#testDeprecatedConfigurations is broken with Hadoop 3.3 (#3320)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-05-28 07:24:48 -07:00
Baiqiang Zhao 21aa553bc1
HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.count` to `hbase.normalizer.merge.min.region.count`
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-24 13:03:27 -07:00
Andrew Purtell 8ec6fd9459
HBASE-25869 WAL value compression (#3244)
WAL storage can be expensive, especially if the cell values
represented in the edits are large, consisting of blobs or
significant lengths of text. Such WALs might need to be kept around
for a fairly long time to satisfy replication constraints on a space
limited (or space-contended) filesystem.

We have a custom dictionary compression scheme for cell metadata that
is engaged when WAL compression is enabled in site configuration.
This is fine for that application, where we can expect the universe
of values and their lengths in the custom dictionaries to be
constrained. For arbitrary cell values it is better to use one of the
available compression codecs, which are suitable for arbitrary albeit
compressible data.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-21 11:05:52 -07:00
Baiqiang Zhao d69d5c24b1
HBASE-25861 Correct the usage of Configuration#addDeprecation (#3249)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-14 09:31:06 -07:00
GeorryHuang 00fec24c90
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-02 21:30:48 +05:30
Duo Zhang 6c65314cdf
HBASE-25819 Fix style issues for StochasticLoadBalancer (#3207)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-29 11:03:55 +08:00
Duo Zhang f36e153964 HBASE-25778 The tracinig implementation for AsyncConnectionImpl.getHbck is incorrect (#3165)
Signed-off-by: meiyi <myimeiyi@gmail.com>
2021-04-25 09:23:23 +08:00
Duo Zhang f6ff519dd0 HBASE-25591 Upgrade opentelemetry to 0.17.1 (#2971)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang bb8c4967f8 HBASE-25535 Set span kind to CLIENT in AbstractRpcClient (#2907)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang 2be2c63f0d HBASE-25484 Add trace support for WAL sync (#2892)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang 03e12bfa4a HBASE-25455 Add trace support for HRegion read/write operation (#2861)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang ae2c62ffaa HBASE-25481 Add host and port attribute when tracing rpc call at client side (#2857)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang 805b2ae2ad HBASE-23898 Add trace support for simple apis in async client (#2813)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang 2420286715 HBASE-25401 Add trace support for async call in rpc client (#2790)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang 302d9ea8b8 HBASE-25373 Remove HTrace completely in code base and try to make use of OpenTelemetry
Signed-off-by: stack <stack@apache.org>
2021-04-25 09:23:23 +08:00
ZhiChen c5b0989d22
HBASE-25762 Improvement for some debug-logging guards (#3145)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-13 23:03:55 +08:00
Duo Zhang f9e928e5a7
HBASE-25184 Move RegionLocationFinder to hbase-balancer (#2543)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-10 21:10:53 +08:00
Jan Hentschel 048ca4e43f
HBASE-25174 Remove deprecated fields in HConstants (#2558)
Remove the deprecated fields, which can be removed in 3.0.0. Marked the
constant OLDEST_TIMESTAMP as InterfaceAudience.Private as it is only use
in classes, which are also marked as InterfaceAudience.Private.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-03 23:12:16 +08:00
Duo Zhang 024248994f
HBASE-25696 Need to initialize SLF4JBridgeHandler in jul-to-slf4j for redirecting jul to slf4j (#3093)
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 15:54:18 +08:00
Duo Zhang ba3610d097
HBASE-19577 Use log4j2 instead of log4j for logging (#1708)
Signed-off-by: stack <stack@apache.org>
2021-03-20 09:21:25 +08:00
haxiaolin 585aca1f05
HBASE-25518 Support separate child regions to different region servers (#3001)
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:38:17 -07:00
bitterfox ebb0adf500
HBASE-25665 Option to use hostname instead of canonical hostname for secure HBase cluster connection (#3051) 2021-03-16 21:04:25 -07:00
shahrs87 7386fb6e1f
HBASE-25622 Result#compareResults should compare tags. (#3026)
Signed-off-by: stack <stack@apache.org>
2021-03-11 21:51:07 -08:00
niuyulin 109bd24065
HBASE-25630 Set switch compaction after bulkload default as false (#3022)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-03-09 12:14:53 +08:00
niuyulin e80b901e47
HBASE-25603 Add switch for compaction after bulkload (#2982)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-04 10:09:08 +08:00
Peter Somogyi b24bd4095a
HBASE-25611 ExportSnapshot chmod flag uses value as decimal (#3003)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-02 11:23:10 +01:00
stack 8a4c3b066b HBASE-25512 May throw StringIndexOutOfBoundsException when construct illegal tablename error #2884
This revert of a revert reapplies the PR; the original application was
missing the HBASE JIRA #; thats why it was reverted and then reapplied
w/ the JIRA # added.

This reverts commit 16fe1e95ec.
2021-02-08 14:35:52 -08:00
stack 16fe1e95ec Revert "fix bug: string out of bounds when construct illegal tablename error message (#2884)"
This reverts commit 098f7c0bd4.
2021-02-08 14:34:19 -08:00
zhengzhuobinzzb 098f7c0bd4
fix bug: string out of bounds when construct illegal tablename error message (#2884)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-02-08 13:38:27 -08:00
Bharath Vissapragada de51a40b53
HBASE-25528: Dedicated merge dispatch threadpool on master (#2904)
Adds "hbase.master.executor.merge.dispatch.threads" and defaults to 2.

Also adds additional logging that includes the number of split plans
and merge plans computed for each normalizer run.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-28 09:33:35 -08:00
Duo Zhang a37e727990
HBASE-25509 ChoreService.cancelChore will not call ScheduledChore.cle… (#2890)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-20 16:10:36 +08:00
申胜利 3488c44a21 HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Revert of the revert -- re-applying HBASE-25449 with a change
of renaming the test hdfs XML configuration file as it was adversely
affecting tests using MiniDFS

This reverts commit c218e576fe.

Co-authored-by: Josh Elser <elserj@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-01-13 20:51:25 -05:00
Duo Zhang c218e576fe Revert "HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml"
This reverts commit 49aba57181.
2021-01-12 16:08:54 +08:00
申胜利 49aba57181 HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Signed-off-by: Josh Elser <elserj@apache.org>
2021-01-08 16:00:44 -05:00
Baiqiang Zhao 8ecfcede96
HBASE-25439 Add BYTE unit in PrettyPrinter.Unit (#2812)
Signed-off-by: stack <stack@apache.org>
2021-01-03 09:34:00 -08:00
申胜利 eef076828a
Shenshengli hbase-25450 The parameter "hbase.bucketcache.size" is misdescribed (#2821)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-01-03 09:27:08 -08:00
Pankaj c96fbf0407
HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone) (#2757)
* HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone)
* HBASE-25379 RemoteProcedureResultReporter also should retry after the configured pause time
* Addressed the review comments

Signed-off-by: Yulin Niu <niuyulin@apache.org>
2020-12-29 22:25:36 +05:30
ramkrish86 140c7f6ea0
HBASE-24850 CellComparator perf improvement (#2802)
* Using ContiguousCellFormat as a marker alone

* Commit the new file

* Fix the comparator logic that was an oversight

* Fix the sequenceId check order

* Adding few more static methods that helps in scan flow like query
matcher where we have more cols

* Remove ContiguousCellFormat and ensure compare() can be inlined

* applying negation as per review comment

* Fix checkstyle comments

* fix review comments

* Address review comments

* Fix the checkstyle issues

* Fix javadoc

Signed-off-by: stack <stack@apache.org>
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-28 13:02:06 +05:30
Michael Stack 56f1dbd116
HBASE-25425 Some notes on RawCell (#2797)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-21 08:08:22 -08:00
Duo Zhang f098461a55 HBASE-25370 Addendum fix checkstyle issue and dependencies 2020-12-12 21:08:51 +08:00
Adam c62c18dca7
HBASE-25370 Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage (#2740)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-10 22:28:21 +08:00
shahrs87 5016219d3c
HBASE-25328 : Add builder method to create Tags
Closes #2707

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-08 18:15:45 +05:30
Andrew Purtell 679dd7b8f9
HBASE-25292 Improve InetSocketAddress usage discipline (#2669)
Network identities should be bound late. Remote addresses should be
resolved at the last possible moment, just before connect(). Network
identity mappings can change, so our code should not inappropriately
cache them. Otherwise we might miss a change and fail to operate normally.

Revert "HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors"
Removes hbase.resolve.hostnames.on.failure and related code. We always
resolve hostnames, as late as possible.

Preserve InetSocketAddress caching per RPC connection. Avoids potential
lookups per Call.

Replace InetSocketAddress with Address where used as a map key. If we want
to key by hostname and/or resolved address we should be explicit about it.
Using Address chooses mapping by hostname and port only.

Add metrics for potential nameservice resolution attempts, whenever an
InetSocketAddress is instantiated for connect; and metrics for failed
resolution, whenever InetSocketAddress#isUnresolved on the new instance
is true.

* Use ServerName directly to build a stub key

* Resolve and cache ISA on a RpcChannel as late as possible, at first call

* Remove now invalid unit test TestCIBadHostname

We resolve DNS at the latest possible time, at first call, and do not
resolve hostnames for creating stubs at all, so this unit test cannot
work now.

Reviewed-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-28 21:01:22 +08:00