Commit Graph

17477 Commits

Author SHA1 Message Date
Duo Zhang 15d27450c2 HBASE-25451 Upgrade commons-io to 2.8.0 (#2825)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-12-31 17:11:49 +08:00
lujiefsi 3166109b91
HBASE-25441 : add security check for some APIs in RSRpcServices (#2831) (#2810)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-31 00:24:34 +05:30
Duo Zhang 9325a775fe HBASE-25452 Use MatcherAssert.assertThat instead of org.junit.Assert.assertThat (#2826)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-12-30 10:03:39 +08:00
Pankaj 382ebf3f4e HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone) (#2757)
* HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone)
* HBASE-25379 RemoteProcedureResultReporter also should retry after the configured pause time
* Addressed the review comments

Signed-off-by: Yulin Niu <niuyulin@apache.org>
(cherry picked from commit c96fbf0407)
2020-12-29 22:33:31 +05:30
lujiefsi e5c6467809
HBASE-25432: branch-2 add security checks for setTableStateInMeta and fixMeta (#2818)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-28 10:58:56 -08:00
ramkrish86 1c41c9732e HBASE-24850 CellComparator perf improvement (#2776)
* Using ContiguousCellFormat as a marker alone

* Commit the new file

* Fix the comparator logic that was an oversight

* Fix the sequenceId check order

* Adding few more static methods that helps in scan flow like query
matcher where we have more cols

* Remove ContiguousCellFormat and ensure compare() can be inlined

* applying negation as per review comment

* Fix checkstyle comments

* fix review comments

* Address review comments

Signed-off-by: stack <stack@apache.org>
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-28 13:08:50 +05:30
Huang Zhuoyue 73b0e8a216 HBASE-25443 Improve the experience of using the Master webpage by change the loading process of snapshot list to asynchronous (#2811)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Ramkrishna S Vasudevan <ramkrishna@apache.org>
2020-12-25 10:21:57 +08:00
stack b07b400b62 HBASE-25425 Some notes on RawCell 2020-12-21 08:13:42 -08:00
Lokesh Khurana 6eee9b1049
HBASE-24620 : Add a ClusterManager which submits command to ZooKeeper and its Agent which picks and execute those Commands (#2299)
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-21 15:36:02 +05:30
scotthunt befa76f55a HBASE-25084 Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-12-21 08:48:14 +00:00
Duo Zhang 1385fb3560 HBASE-25420 Some minor improvements in rpc implementation (#2792)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
2020-12-20 11:30:16 +08:00
Akshay Sudheer 914b356d15 HBASE-25404 Procedures table Id under master web UI gets word break to single character (#2783)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-19 12:38:39 -08:00
huaxiangsun 8159fc8dbe
HBASE-25293 Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side. (#2768) (#2786)
Signed-off-by: stack <stack@apache.org>
2020-12-18 09:25:04 -08:00
Sandeep Pal aa5a477426
HBASE-25383: Ability to update and remove peer base config
Closes #2778

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-18 14:20:36 +05:30
huaxiangsun 22dd0545b9
HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName (#2759)
Signed-off-by: stack <stack@apache.org>
2020-12-17 15:21:30 -08:00
haxl 32c4432b9b
HBASE-25334 TestRSGroupsFallback.testFallback is flaky (#2775)
Signed-off-by: stack <stack@apache.org>
2020-12-16 13:21:53 -08:00
stack 899a0d8654 HBASE-25400 [Flakey Tests] branch-2 TestRegionMoveAndAbandon 2020-12-16 10:35:34 -08:00
Baiqiang Zhao b6c03f2027
HBASE-25365 The log in move_servers_rsgroup is incorrect (#2743)
Signed-off-by: stack <stack@apache.org>
2020-12-16 09:13:57 -08:00
shahrs87 9b4be47c24
HBASE-25246 Backup/Restore hbase cell tags (#2766)
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-16 18:59:19 +05:30
Pankaj 50890acda7 HBASE-25378 Legacy comparator in Hfile trailer will fail to load (#2756)
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 9bdac6cd17)
2020-12-15 23:59:18 +05:30
Michael Stack aebcb2f192
HBASE-25389 [Flakey Tests] branch-2 TestMetaShutdownHandler (#2773)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-12-15 09:49:16 -08:00
Adam 9c2f1f7a2b HBASE-25370 Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage (#2740)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-11 08:23:49 +08:00
haxl 210d8a7b56 HBASE-25287 Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when loading files (#2699)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-10 22:21:14 +08:00
lixiaobao 81868e328a HBASE-25372 Fix typo in ban-jersey section of the enforcer plugin in pom.xml (#2749)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-09 21:42:29 +08:00
Pankaj 6ff02dedf8
HBASE-25277 postScannerFilterRow impacts Scan performance a lot in HBase 2.x (#2751)
1. Added a check for Object class in RegionCoprocessorHost to avoid wrong initialization of hasCustomPostScannerFilterRow
2. Removed dummy implementation of postScannerFilterRow from AccessController, VisibilityController & ConstraintProcessor (which are not required currently)

Signed-off-by Ramkrishna S Vasudevan <ramkrishna@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-09 10:39:44 +05:30
haxiaolin e855b627e0 HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWriteLock instead of synchronize
Closes #2738

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2020-12-08 22:00:51 +08:00
shahrs87 65ac1dbeb0
HBASE-25328 : Add builder method to create Tags
Closes #2707

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-08 18:17:30 +05:30
Duo Zhang ec47e46c7e HBASE-25336 Use Address instead of InetSocketAddress in RpcClient implementation (#2716)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-12-07 21:50:16 +08:00
stack 7bb3ab7687 HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster 2020-12-05 14:21:52 -08:00
stack 4d695ad74a HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Addendum; Reset counter so we actually wait in the new loop added by the
above.
2020-12-05 14:00:18 -08:00
Michael Stack a4d5b1bb88
HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Add a bit of a wait before testing if online replicas match the zk count. It might take a
while for all replicas to come online.

Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-05 08:55:59 -08:00
Andrew Purtell ea331a66be HBASE-25292 Improve InetSocketAddress usage discipline (#2669)
Network identities should be bound late. Remote addresses should be
resolved at the last possible moment, just before connect(). Network
identity mappings can change, so our code should not inappropriately
cache them. Otherwise we might miss a change and fail to operate normally.

Revert "HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors"
Removes hbase.resolve.hostnames.on.failure and related code. We always
resolve hostnames, as late as possible.

Preserve InetSocketAddress caching per RPC connection. Avoids potential
lookups per Call.

Replace InetSocketAddress with Address where used as a map key. If we want
to key by hostname and/or resolved address we should be explicit about it.
Using Address chooses mapping by hostname and port only.

Add metrics for potential nameservice resolution attempts, whenever an
InetSocketAddress is instantiated for connect; and metrics for failed
resolution, whenever InetSocketAddress#isUnresolved on the new instance
is true.

* Use ServerName directly to build a stub key

* Resolve and cache ISA on a RpcChannel as late as possible, at first call

* Remove now invalid unit test TestCIBadHostname

We resolve DNS at the latest possible time, at first call, and do not
resolve hostnames for creating stubs at all, so this unit test cannot
work now.

Reviewed-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-04 10:19:02 -08:00
Viraj Jasani a95c0c6537
Revert "HBASE-25246 Backup/Restore hbase cell tags"
This reverts commit 34721c42ec.
2020-12-03 21:17:11 +05:30
tianhang 8c43d3bc09 HBASE-25355 [Documentation] Fix spelling error (#2735)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-03 21:36:32 +08:00
Laxman Goswami 5e683e1962
HBASE-25230 Embedded zookeeper server not clean up the old data
Closes #2732

Signed-off-by: maoling <maoling199210191@sina.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-03 14:45:14 +05:30
Michael Stack 9cc2f76248
HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.testReplicationRefreshSource:141 Waiting timed out after [60,000] msec (#2731)
Start the check for recovered queue presence earlier.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-12-02 09:55:24 -08:00
shahrs87 34721c42ec
HBASE-25246 Backup/Restore hbase cell tags
Closes #2706

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-02 21:30:46 +05:30
Duo Zhang 90e96d1a6e HBASE-25342 Upgrade error prone to 2.4.0 (#2725)
Have to disable MutablePublicArray because of a bug in error prone
https://github.com/google/error-prone/issues/1645

Signed-off-by: stack <stack@apache.org>
2020-12-02 23:38:05 +08:00
stack 745a3a9ab7 Revert "Revert "HBASE-25320 Upgrade hbase-thirdparty dependency to 3.4.1 (#2693)""
This reverts commit 78f30ff496.

This is a revert of a revert; i.e. a repplication -- original patch was
good.
2020-12-01 17:35:43 -08:00
stack 78f30ff496 Revert "HBASE-25320 Upgrade hbase-thirdparty dependency to 3.4.1 (#2693)"
This reverts commit 0851e3273a.
2020-12-01 09:35:51 -08:00
Duo Zhang 0851e3273a HBASE-25320 Upgrade hbase-thirdparty dependency to 3.4.1 (#2693)
Signed-off-by: stack <stack@apache.org>
2020-12-01 08:17:17 -08:00
lujiefsi a1db679e4d
HBASE-25332:fix One pontential NPE
Closes #2715

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-01 19:47:09 +05:30
clarax 4c8fa47b28
HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2655)
* HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)


Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-11-30 22:51:57 -08:00
Michael Stack f0a4aefcdf
HBASE-25345 [Flakey Tests] branch-2 TestReadReplicas#testVerifySecondaryAbilityToReadWithOnFiles (#2727)
Check TEST_SKIP_REPORTING_TRANSITION and if true, skip trying to talk to
update master on state transition -- i.e. reportFileArchivalForQuotas --
as we allow for reportRegionStateTransition (For tests only)

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-11-30 21:02:45 -08:00
stack 01337e4104 Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) (#2682)"
This reverts commit 5fcbf1f9f2.
2020-11-30 16:45:59 -08:00
Michael Stack 5fcbf1f9f2 HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644) (#2682)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-30 10:29:29 -08:00
GeorryHuang 433235e98c
HBASE-25339 Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure
Closes #2717

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-30 19:50:54 +05:30
Viraj Jasani 904b8a814b
Revert "HBASE-25339 Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure"
This reverts commit 861c5509ad.
2020-11-30 19:50:29 +05:30
GeorryHuang 861c5509ad
HBASE-25339 Method parameter and member variable are duplicated in checkSplittable() of SplitTableRegionProcedure
Closes #2717

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-30 18:01:05 +05:30
Balazs Meszaros b75b772bc6
HBASE-25307 ThreadLocal pooling leads to NullPointerException (#2709)
* PoolMap does not discard any elements anymore. If an element is put,
it always stores it. The reason: it stores expensive resources (rpc
connections) which would lead to resource leak if we simple discard it.
RpcClients can reference netty ByteBufs which are reference counted.
Resource cleanup is done by AbstractRpcClient.cleanupIdleConnections().
* PoolMap does not implement Map interface anymore, so ensuring
thread-safety has become easier. Put method is replaced with getOrCreate().
* ThreadLocalPool doesn't use ThreadLocal class anymore. It stores
resources on thread basis, but it doesn't remove values when a thread
exits. Again, proper cleanup is done by cleanupIdleConnections().

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
2020-11-30 12:36:03 +01:00