Commit Graph

8581 Commits

Author SHA1 Message Date
leyangyueshan 043da5f5ee
HBASE-25435 Slow metric value can be configured (#2823)
* HBASE-25435 Slow metric value can be configured

* fix  HBASE-25435

* hbase-25435 add blank

* fix hbase-25435-2

Co-authored-by: stevenxi <stevenxi@tencent.com>
2020-12-31 22:37:31 +08:00
Duo Zhang fbf00f9c28
HBASE-25451 Upgrade commons-io to 2.8.0 (#2825)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-12-31 16:57:27 +08:00
lujiefsi 29a9a16de4
HBASE-25441 : add security check for some APIs in RSRpcServices (#2810)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-31 00:22:26 +05:30
Duo Zhang 55a4eca9e6
HBASE-25452 Use MatcherAssert.assertThat instead of org.junit.Assert.assertThat (#2826)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-12-30 09:47:56 +08:00
Pankaj c96fbf0407
HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone) (#2757)
* HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone)
* HBASE-25379 RemoteProcedureResultReporter also should retry after the configured pause time
* Addressed the review comments

Signed-off-by: Yulin Niu <niuyulin@apache.org>
2020-12-29 22:25:36 +05:30
lujiefsi d963342f8a
HBASE-25432:add security checks for setTableStateInMeta and fixMeta (#2809)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-28 10:57:30 -08:00
ramkrish86 140c7f6ea0
HBASE-24850 CellComparator perf improvement (#2802)
* Using ContiguousCellFormat as a marker alone

* Commit the new file

* Fix the comparator logic that was an oversight

* Fix the sequenceId check order

* Adding few more static methods that helps in scan flow like query
matcher where we have more cols

* Remove ContiguousCellFormat and ensure compare() can be inlined

* applying negation as per review comment

* Fix checkstyle comments

* fix review comments

* Address review comments

* Fix the checkstyle issues

* Fix javadoc

Signed-off-by: stack <stack@apache.org>
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-28 13:02:06 +05:30
Huang Zhuoyue 0f868da05d HBASE-25443 Improve the experience of using the Master webpage by change the loading process of snapshot list to asynchronous 2020-12-25 10:09:32 +08:00
Mohammad Arshad dcb38f47db
HBASE-25371: When openRegion fails during initial verification(before… (#2785)
Signed-off-by: stack <stack@apache.org>
2020-12-21 08:11:22 -08:00
scotthunt 904b555edc HBASE-25084 Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-12-21 08:44:33 +00:00
Wellington Chevreuil d13010fa26 Revert "[PATCH] Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784)"
Will re-commit with the proper jira ID

This reverts commit 51cee0016f.
2020-12-21 08:41:52 +00:00
scotthunt 51cee0016f
[PATCH] Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-12-21 08:39:13 +00:00
Duo Zhang 772350b26e
HBASE-25419 Remove deprecated methods in RpcServer implementation (#2791)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
2020-12-20 12:00:07 +08:00
Duo Zhang 1540b89cee
HBASE-25420 Some minor improvements in rpc implementation (#2792)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
2020-12-20 11:26:36 +08:00
Akshay Sudheer 7b1e9cd0d2
HBASE-25404 Procedures table Id under master web UI gets word break to single character (#2783)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-19 12:37:21 -08:00
Duo Zhang 33441f540a Revert "HBASE-25368 Filter out more invalid encoded name in isEncodedRegionName(byte[] regionName) (#2753)"
This reverts commit c327680125.
2020-12-19 20:28:30 +08:00
Sandeep Pal f600856a3b
HBASE-25383: Ability to update and remove peer base config
Closes #2778

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-18 14:19:17 +05:30
stack 888a4fb86a HBASE-25400 [Flakey Tests] branch-2 TestRegionMoveAndAbandon 2020-12-16 10:39:11 -08:00
shahrs87 74d68180e6
HBASE-25246 Backup/Restore hbase cell tags
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-16 18:52:36 +05:30
huaxiangsun c327680125
HBASE-25368 Filter out more invalid encoded name in isEncodedRegionName(byte[] regionName) (#2753)
Signed-off-by: stack <stack@apache.com>
2020-12-15 21:52:54 -08:00
huaxiangsun 1c217da2ff
HBASE-25293 Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side. (#2768)
Signed-off-by: stack <stack@apache.org>
2020-12-15 21:45:39 -08:00
Baiqiang Zhao 1bb9b78787
HBASE-25365 The log in move_servers_rsgroup is incorrect (#2742)
Signed-off-by: stack <stack@apache.or>
2020-12-15 20:25:59 -08:00
Pankaj 9bdac6cd17
HBASE-25378 Legacy comparator in Hfile trailer will fail to load (#2756)
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-15 23:56:02 +05:30
Michael Stack a4d42d1fcc HBASE-25389 [Flakey Tests] branch-2 TestMetaShutdownHandler (#2773)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-12-15 09:51:56 -08:00
Bo Cui d50816fe44
HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl… (#2739)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-12 21:10:33 +08:00
Huaxiang Sun e04792959a Revert "HBASE-25293 Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side."
This reverts commit c1aa3b24e9.
2020-12-11 10:22:41 -08:00
haxl 635c911532
HBASE-25287 Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when loading files (#2699)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-10 22:15:39 +08:00
haxl a9b8c10f80
HBASE-25334 TestRSGroupsFallback.testFallback is flaky (#2728)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-10 22:07:06 +08:00
Huaxiang Sun c1aa3b24e9 HBASE-25293 Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side. 2020-12-09 09:12:01 -08:00
haxiaolin 56dd3eba81 HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWriteLock instead of synchronize 2020-12-08 21:09:31 +08:00
Pankaj fb6e498b32
HBASE-25277 postScannerFilterRow impacts Scan performance a lot in HBase 2.x (#2675)
* HBASE-25277 postScannerFilterRow impacts Scan performance a lot in HBase 2.x

1. Added a check for Object class in RegionCoprocessorHost to avoid wrong initialization of hasCustomPostScannerFilterRow
2. Removed dummy implementation of postScannerFilterRow from AccessController, VisibilityController & ConstraintProcessor (which are not required currently)

Signed-off-by Ramkrishna S Vasudevan <ramkrishna@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-07 23:00:48 +05:30
stack 7d0a687e57 HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Addendum; Reset counter so we actually wait in the new loop added by the
above.
2020-12-05 14:01:56 -08:00
Michael Stack 6791b29771 HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Add a bit of a wait before testing if online replicas match the zk count. It might take a
while for all replicas to come online.

Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-05 08:56:44 -08:00
Michael Stack 946fa81715 HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.testReplicationRefreshSource:141 Waiting timed out after [60,000] msec (#2731)
Start the check for recovered queue presence earlier.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-12-02 09:58:12 -08:00
lujiefsi 8938b7a678
HBASE-25332:fix One pontential NPE
Closes #2715

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-01 19:41:38 +05:30
Duo Zhang 86bb037eb0 Revert "HBASE-25334 TestRSGroupsFallback.testFallback is flaky (#2712)"
This reverts commit f71eb27be1.
2020-12-01 16:22:09 +08:00
haxl f71eb27be1
HBASE-25334 TestRSGroupsFallback.testFallback is flaky (#2712)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-01 15:24:02 +08:00
Michael Stack 7da90a1e23 HBASE-25345 [Flakey Tests] branch-2 TestReadReplicas#testVerifySecondaryAbilityToReadWithOnFiles (#2727)
Check TEST_SKIP_REPORTING_TRANSITION and if true, skip trying to talk to
update master on state transition -- i.e. reportFileArchivalForQuotas --
as we allow for reportRegionStateTransition (For tests only)

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-11-30 21:04:51 -08:00
GeorryHuang 264412d30c
HBASE-25339 Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure
Closes #2717

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-30 19:49:46 +05:30
Viraj Jasani a8a35da73e
Revert "HBASE-25339 Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure"
This reverts commit c76145c505.
2020-11-30 19:48:58 +05:30
GeorryHuang c76145c505
HBASE-25339 Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure
Closes #2717

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-30 17:59:45 +05:30
Duo Zhang fc62214e77
HBASE-25341 Fix ErrorProne error which causes nightly to fail (#2718)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-30 14:14:44 +08:00
Akshay Sudheer bfa5cb7d48
HBASE-25321 The sort icons not shown after Upgrade JQuery to 3.5.1 (#2694)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-29 12:08:57 +08:00
Baiqiang Zhao e2c1b10cb7
HBASE-25330 RSGroupInfoManagerImpl#moveServers return is not set of servers moved (#2708)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-29 11:32:51 +08:00
Andrew Purtell 679dd7b8f9
HBASE-25292 Improve InetSocketAddress usage discipline (#2669)
Network identities should be bound late. Remote addresses should be
resolved at the last possible moment, just before connect(). Network
identity mappings can change, so our code should not inappropriately
cache them. Otherwise we might miss a change and fail to operate normally.

Revert "HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors"
Removes hbase.resolve.hostnames.on.failure and related code. We always
resolve hostnames, as late as possible.

Preserve InetSocketAddress caching per RPC connection. Avoids potential
lookups per Call.

Replace InetSocketAddress with Address where used as a map key. If we want
to key by hostname and/or resolved address we should be explicit about it.
Using Address chooses mapping by hostname and port only.

Add metrics for potential nameservice resolution attempts, whenever an
InetSocketAddress is instantiated for connect; and metrics for failed
resolution, whenever InetSocketAddress#isUnresolved on the new instance
is true.

* Use ServerName directly to build a stub key

* Resolve and cache ISA on a RpcChannel as late as possible, at first call

* Remove now invalid unit test TestCIBadHostname

We resolve DNS at the latest possible time, at first call, and do not
resolve hostnames for creating stubs at all, so this unit test cannot
work now.

Reviewed-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-28 21:01:22 +08:00
Lokesh Khurana e6165412e0
HBASE-25237 : 'hbase master stop' shuts down the cluster, not the master only
Closes #2705

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-27 16:02:30 +05:30
Qi Yu dc48071e01
HBASE-25324 Remove unnecessary array to list conversion in SplitLogManager (#2703)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.com>
2020-11-27 08:20:24 +08:00
Mate Szalay-Beko 451a4b06b1 HBASE-25263 Various improvements of column family encryption
This PR is a follow-up of HBASE-25181 (#2539), where several issues were
discussed on the PR:

1. Currently we use PBKDF2WithHmacSHA1 key generation algorithm to generate a
secret key for HFile / WalFile encryption, when the user is defining a string
encryption key in the hbase shell. This algorithm is not secure enough and
not allowed in certain environments (e.g. on FIPS compliant clusters). We are
changing it to PBKDF2WithHmacSHA384. It will not break backward-compatibility,
as even the tables created by the shell using the new algorithm will be able
to load (e.g. during bulkload / replication) the HFiles serialized with the
key generated by an old algorithm, as the HFiles themselves already contain
the key necessary for their decryption.

Smaller issues fixed by this commit:

2. Improve the documentation e.g. with the changes introduced by HBASE-25181
and also by some points discussed on the Jira ticket of HBASE-25263.

3. In EncryptionUtil.createEncryptionContext the various encryption config
checks should throw IllegalStateExceptions instead of RuntimeExceptions.

4. Test cases in TestEncryptionTest.java should be broken down into smaller
tests.

5. TestEncryptionDisabled.java should use ExpectedException JUnit rule to
validate exceptions.

closes #2676

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-11-26 16:44:34 +01:00
Toshihiro Suzuki b142f5dcd2
HBASE-25242 Add Increment/Append support to RowMutations (#2630)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-11-26 13:31:43 +09:00
Andrew Purtell 1726160839
HBASE-24640 Purge use of VisibleForTesting (#2695)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-11-25 14:10:12 -08:00