8179 Commits

Author SHA1 Message Date
shahrs87
2c72225d13
HBASE-25536 Remove 0 length wal file from logQueue if it belongs to old sources (#2908)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-29 17:49:50 +05:30
Bharath Vissapragada
b930ea1ebb
HBASE-25528: Dedicated merge dispatch threadpool on master (#2910)
Adds "hbase.master.executor.merge.dispatch.threads" and defaults to 2.

Also adds additional logging that includes the number of split plans
and merge plans computed for each normalizer run.

(cherry picked from commit 36b4698cea9485f4d491612342b8c841e1c3db86)

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-01-28 16:09:18 -08:00
Aman Poonia
818000de7b
HBASE-25523 Region normalizer chore thread is getting killed (#2903)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-26 11:56:39 +05:30
Duo Zhang
423ffb8a76 HBASE-25509 fix typo when resolving conflicts 2021-01-20 23:35:49 +08:00
Duo Zhang
f70229b193 HBASE-25509 ChoreService.cancelChore will not call ScheduledChore.cle… (#2890)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-20 23:24:16 +08:00
Christine Feng
854f75982f
HBASE-25478 : Implement retries when enabling tables in TestRegionReplicaReplicationEndpoint (#2896)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-20 11:32:43 +05:30
Anjan Das
fbc44784db
HBASE-25475 : Unset zk based wal splitting explicitly in tests (ADDENDUM) (#2891)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-17 16:34:25 +05:30
Anjan Das
24739256aa
HBASE-25475: Improve UT added as part of HBASE-25445 in TestSplitWALManager (#2855)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-01-15 12:29:23 +05:30
Baiqiang Zhao
f391a2b865 HBASE-25431 MAX_FILESIZE and MEMSTORE_FLUSHSIZE should not be set negative number (#2803)
Signed-off-by: stack <stack@apache.org>
2021-01-14 22:01:50 -08:00
Tak Lon (Stephen) Wu
2bb7beb448
Backport "HBASE-25249 Adding StoreContext" to branch-2 (#2869)
Co-authored-by: Abhishek Khanna <akkhanna@amazon.com>

Signed-off-by: Zach York <zyork@apache.org>
2021-01-13 20:13:43 -08:00
Bo Cui
1612b9ef85
HBASE-23340 hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWALs too large (more than 2TB) (#2779)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
    Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2021-01-14 09:06:07 +05:30
caroliney14
f4ad0d0988
HBASE-25329 Dump ritsOverThreshold in logs (#2761)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-13 12:13:05 +05:30
Viraj Jasani
b817a156c9 HBASE-25211 : Rack awareness in RegionMover (#2795)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-01-12 15:32:31 -08:00
Wellington Ramos Chevreuil
fdae12d754
HBASE-24813 ReplicationSource should clear buffer usage on Replicatio… (#2546) (#2849)
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-01-12 09:40:25 +00:00
xijiawen
6a401d89f8 HBASE-25459 WAL can't be cleaned in some scenes (#2848)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-01-11 10:01:53 +00:00
Bo Cui
bd8ded8d60 HBASE-25483 set the loadMeta log level to debug (#2859)
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit a348204d4a77c0e15ddfd459cecfc4d3427d0a63)
2021-01-08 19:30:17 +05:30
xijiawen
4705b4752b HBASE-25434 SlowDelete & SlowPut metric value should use updateDelete & updatePut (#2837)
Signed-off-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 7d70141b90d188d69e259ad5de435dad4eed5bf8)
2021-01-08 19:04:32 +08:00
Duo Zhang
a573e49d69 HBASE-25471 Move RegionScannerImpl out of HRegion (#2853)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-01-08 17:57:24 +08:00
Anjan Das
d93588512a
HBASE-25445: Use WAL FS instead of master FS in SplitWALManager (#2844)
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-07 15:33:12 +05:30
Duo Zhang
1719565aa7 HBASE-25458 HRegion methods cleanup (#2838)
Signed-off-by: meiyi <myimeiyi@gmail.com>
2021-01-07 10:18:41 +08:00
Bo Cui
ae77f81e7e HBASE-25447 remoteProc is suspended due to OOM ERROR (#2824)
Some OMME can not cause the JVM to exit, like "java.lang.OutOfMemoryError: Direct buffer memory", "java.lang.OutOfMemoryError: unable to create new native thread", as they dont call vmError#next_OnError_command. So abort HMaster when uncaught exception occurs in TimeoutExecutor, the new active Hmaster will resume the suspended procedure.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit 600be60a4bd4d3b3e9652027a0cb8bdd32016c6b)
2021-01-04 21:12:50 +05:30
leyangyueshan
e517bf7058 HBASE-25435 Slow metric value can be configured
Signed-off-by: jediwang <jediwang@tencent.com>
2021-01-04 13:14:57 +08:00
申胜利
d2c1886bf4 Shenshengli hbase-25450 The parameter "hbase.bucketcache.size" is misdescribed (#2821)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-01-03 09:31:10 -08:00
GeorryHuang
605240f8a9 HBASE-24751 Display Task completion time and/or processing duration on Web UI (#2815)
Signed-off-by: stack <stack@apache.org>
2021-01-02 11:25:31 -08:00
lujiefsi
d0221083ca
HBASE-25456 : add security check for setRegionStateInMeta (#2834) (#2833)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-01 12:18:34 +05:30
Duo Zhang
15d27450c2 HBASE-25451 Upgrade commons-io to 2.8.0 (#2825)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-12-31 17:11:49 +08:00
lujiefsi
3166109b91
HBASE-25441 : add security check for some APIs in RSRpcServices (#2831) (#2810)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-31 00:24:34 +05:30
Duo Zhang
9325a775fe HBASE-25452 Use MatcherAssert.assertThat instead of org.junit.Assert.assertThat (#2826)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-12-30 10:03:39 +08:00
Pankaj
382ebf3f4e HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone) (#2757)
* HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone)
* HBASE-25379 RemoteProcedureResultReporter also should retry after the configured pause time
* Addressed the review comments

Signed-off-by: Yulin Niu <niuyulin@apache.org>
(cherry picked from commit c96fbf04077fa37555033f88939fdd69ac810b35)
2020-12-29 22:33:31 +05:30
lujiefsi
e5c6467809
HBASE-25432: branch-2 add security checks for setTableStateInMeta and fixMeta (#2818)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-28 10:58:56 -08:00
ramkrish86
1c41c9732e HBASE-24850 CellComparator perf improvement (#2776)
* Using ContiguousCellFormat as a marker alone

* Commit the new file

* Fix the comparator logic that was an oversight

* Fix the sequenceId check order

* Adding few more static methods that helps in scan flow like query
matcher where we have more cols

* Remove ContiguousCellFormat and ensure compare() can be inlined

* applying negation as per review comment

* Fix checkstyle comments

* fix review comments

* Address review comments

Signed-off-by: stack <stack@apache.org>
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-28 13:08:50 +05:30
Huang Zhuoyue
73b0e8a216 HBASE-25443 Improve the experience of using the Master webpage by change the loading process of snapshot list to asynchronous (#2811)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Ramkrishna S Vasudevan <ramkrishna@apache.org>
2020-12-25 10:21:57 +08:00
scotthunt
befa76f55a HBASE-25084 Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-12-21 08:48:14 +00:00
Duo Zhang
1385fb3560 HBASE-25420 Some minor improvements in rpc implementation (#2792)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
2020-12-20 11:30:16 +08:00
Akshay Sudheer
914b356d15 HBASE-25404 Procedures table Id under master web UI gets word break to single character (#2783)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-19 12:38:39 -08:00
huaxiangsun
8159fc8dbe
HBASE-25293 Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side. (#2768) (#2786)
Signed-off-by: stack <stack@apache.org>
2020-12-18 09:25:04 -08:00
Sandeep Pal
aa5a477426
HBASE-25383: Ability to update and remove peer base config
Closes #2778

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-18 14:20:36 +05:30
huaxiangsun
22dd0545b9
HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName (#2759)
Signed-off-by: stack <stack@apache.org>
2020-12-17 15:21:30 -08:00
stack
899a0d8654 HBASE-25400 [Flakey Tests] branch-2 TestRegionMoveAndAbandon 2020-12-16 10:35:34 -08:00
shahrs87
9b4be47c24
HBASE-25246 Backup/Restore hbase cell tags (#2766)
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-16 18:59:19 +05:30
Pankaj
50890acda7 HBASE-25378 Legacy comparator in Hfile trailer will fail to load (#2756)
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 9bdac6cd174bce570a2faf92849931b3e76f240d)
2020-12-15 23:59:18 +05:30
Michael Stack
aebcb2f192
HBASE-25389 [Flakey Tests] branch-2 TestMetaShutdownHandler (#2773)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-12-15 09:49:16 -08:00
haxl
210d8a7b56 HBASE-25287 Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when loading files (#2699)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-10 22:21:14 +08:00
Pankaj
6ff02dedf8
HBASE-25277 postScannerFilterRow impacts Scan performance a lot in HBase 2.x (#2751)
1. Added a check for Object class in RegionCoprocessorHost to avoid wrong initialization of hasCustomPostScannerFilterRow
2. Removed dummy implementation of postScannerFilterRow from AccessController, VisibilityController & ConstraintProcessor (which are not required currently)

Signed-off-by Ramkrishna S Vasudevan <ramkrishna@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-09 10:39:44 +05:30
haxiaolin
e855b627e0 HBASE-25363 Improve performance of HFileLinkCleaner by using ReadWriteLock instead of synchronize
Closes #2738

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2020-12-08 22:00:51 +08:00
stack
4d695ad74a HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Addendum; Reset counter so we actually wait in the new loop added by the
above.
2020-12-05 14:00:18 -08:00
Michael Stack
a4d5b1bb88
HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Add a bit of a wait before testing if online replicas match the zk count. It might take a
while for all replicas to come online.

Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-05 08:55:59 -08:00
Andrew Purtell
ea331a66be HBASE-25292 Improve InetSocketAddress usage discipline (#2669)
Network identities should be bound late. Remote addresses should be
resolved at the last possible moment, just before connect(). Network
identity mappings can change, so our code should not inappropriately
cache them. Otherwise we might miss a change and fail to operate normally.

Revert "HBASE-14544 Allow HConnectionImpl to not refresh the dns on errors"
Removes hbase.resolve.hostnames.on.failure and related code. We always
resolve hostnames, as late as possible.

Preserve InetSocketAddress caching per RPC connection. Avoids potential
lookups per Call.

Replace InetSocketAddress with Address where used as a map key. If we want
to key by hostname and/or resolved address we should be explicit about it.
Using Address chooses mapping by hostname and port only.

Add metrics for potential nameservice resolution attempts, whenever an
InetSocketAddress is instantiated for connect; and metrics for failed
resolution, whenever InetSocketAddress#isUnresolved on the new instance
is true.

* Use ServerName directly to build a stub key

* Resolve and cache ISA on a RpcChannel as late as possible, at first call

* Remove now invalid unit test TestCIBadHostname

We resolve DNS at the latest possible time, at first call, and do not
resolve hostnames for creating stubs at all, so this unit test cannot
work now.

Reviewed-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-04 10:19:02 -08:00
Michael Stack
9cc2f76248
HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.testReplicationRefreshSource:141 Waiting timed out after [60,000] msec (#2731)
Start the check for recovered queue presence earlier.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-12-02 09:55:24 -08:00
Michael Stack
f0a4aefcdf
HBASE-25345 [Flakey Tests] branch-2 TestReadReplicas#testVerifySecondaryAbilityToReadWithOnFiles (#2727)
Check TEST_SKIP_REPORTING_TRANSITION and if true, skip trying to talk to
update master on state transition -- i.e. reportFileArchivalForQuotas --
as we allow for reportRegionStateTransition (For tests only)

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-11-30 21:02:45 -08:00