17588 Commits

Author SHA1 Message Date
Duo Zhang
43ee12640b Revert "HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml"
This reverts commit 09db88d3ba93962c5c97a4947ffe82c2d54fffe7.
2021-01-12 16:10:41 +08:00
xijiawen
aa27488268 HBASE-25459 WAL can't be cleaned in some scenes (#2848)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 6a401d89f89c4c394ea7f4282edcd07afbac167a)
2021-01-11 10:06:09 +00:00
Duo Zhang
75f5cb2450 HBASE-25333 Add maven enforcer rule to ban VisibleForTesting imports (#2854)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-01-09 08:56:09 +08:00
申胜利
09db88d3ba HBASE-25449 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
Signed-off-by: Josh Elser <elserj@apache.org>
2021-01-08 16:01:47 -05:00
Bo Cui
8b48c60347 HBASE-25483 set the loadMeta log level to debug (#2859)
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit a348204d4a77c0e15ddfd459cecfc4d3427d0a63)
2021-01-08 19:32:48 +05:30
xijiawen
99f2b71cc9 HBASE-25434 SlowDelete & SlowPut metric value should use updateDelete & updatePut (#2837)
Signed-off-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 7d70141b90d188d69e259ad5de435dad4eed5bf8)
2021-01-08 19:02:59 +08:00
Anjan Das
e39bae4ddb
HBASE-25445: Use WAL FS instead of master FS in SplitWALManager (#2844)
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-07 15:38:52 +05:30
Bo Cui
82a63abc8a HBASE-25447 remoteProc is suspended due to OOM ERROR (#2824)
Some OMME can not cause the JVM to exit, like "java.lang.OutOfMemoryError: Direct buffer memory", "java.lang.OutOfMemoryError: unable to create new native thread", as they dont call vmError#next_OnError_command. So abort HMaster when uncaught exception occurs in TimeoutExecutor, the new active Hmaster will resume the suspended procedure.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit 600be60a4bd4d3b3e9652027a0cb8bdd32016c6b)
(cherry picked from commit ae77f81e7eaf07e9943688f10f7f7e14211151c2)
2021-01-06 23:46:24 +05:30
mokai
0a4685b085
HBASE-24755 [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup (#2847) (#2846)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-01-06 20:00:45 +05:30
Gary Wang
3f2b9b37a7
HBASE-25463 fix comment error of append.rb (#2845)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-05 20:11:16 +05:30
Mate Szalay-Beko
accbfa0a21 HBASE-25318 Config option for IntegrationTestImportTsv where to generate HFiles to bulkload (#2777)
IntegrationTestImportTsv is generating HFiles under the working directory of the
current hdfs user executing the tool, before bulkloading it into HBase.

Assuming you encrypt the HBase root directory within HDFS (using HDFS
Transparent Encryption), you can bulkload HFiles only if they sit in the same
encryption zone in HDFS as the HBase root directory itself.

When IntegrationTestImportTsv is executed against a real distributed cluster
and the working directory of the current user (e.g. /user/hbase) is not in the
same encryption zone as the HBase root directory (e.g. /hbase/data) then you
will get an exception:

```
ERROR org.apache.hadoop.hbase.regionserver.HRegion: There was a partial failure
due to IO when attempting to load d :
hdfs://mycluster/user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/
IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc

org.apache.hadoop.ipc.RemoteException(java.io.IOException):
/user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/
IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc
can't be moved into an encryption zone.
```

In this commit I make it configurable where the IntegrationTestImportTsv
generates the HFiles.

Co-authored-by: Mate Szalay-Beko <symat@apache.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-01-05 10:33:01 +01:00
Duo Zhang
0e420588b6 HBASE-25457 Possible race in AsyncConnectionImpl between getChoreServ… (#2839)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-04 23:52:45 +08:00
leyangyueshan
35648278fc HBASE-25435 Slow metric value can be configured
Signed-off-by: jediwang <jediwang@tencent.com>
2021-01-04 13:06:55 +08:00
lujiefsi
f5a7fffd8b
HBASE-25456 : add security check for setRegionStateInMeta (#2835) (#2833)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-01-01 12:19:25 +05:30
lujiefsi
a14f08876b
HBASE-25441 : add security check for some APIs in RSRpcServices (#2830) (#2810)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-31 00:25:49 +05:30
Duo Zhang
90db5e4f99 HBASE-25452 Use MatcherAssert.assertThat instead of org.junit.Assert.assertThat (#2826)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-12-30 11:06:58 +08:00
lujiefsi
c3d755e318
HBASE-25432: branch-2.4 add security checks for setTableStateInMeta and fixMeta (#2817)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-28 10:58:31 -08:00
ramkrish86
e7307c91d0 HBASE-24850 CellComparator perf improvement (#2776)
* Using ContiguousCellFormat as a marker alone

* Commit the new file

* Fix the comparator logic that was an oversight

* Fix the sequenceId check order

* Adding few more static methods that helps in scan flow like query
matcher where we have more cols

* Remove ContiguousCellFormat and ensure compare() can be inlined

* applying negation as per review comment

* Fix checkstyle comments

* fix review comments

* Address review comments

Signed-off-by: stack <stack@apache.org>
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-28 13:12:38 +05:30
stack
eedd976c6e HBASE-25425 Some notes on RawCell 2020-12-21 08:14:10 -08:00
Lokesh Khurana
9229a8b986
HBASE-24620 : Add a ClusterManager which submits command to ZooKeeper and its Agent which picks and execute those Commands (#2299)
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-21 15:41:37 +05:30
Duo Zhang
dede90dc54 HBASE-25420 Some minor improvements in rpc implementation (#2792)
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
2020-12-20 11:30:35 +08:00
huaxiangsun
baf3122ad8
HBASE-25356 HBaseAdmin#getRegion() needs to filter out non-regionName and non-encodedRegionName (#2759) (#2789)
Signed-off-by: stack <stack@apache.org>
2020-12-18 13:16:03 -08:00
huaxiangsun
1932a58a62
HBASE-25293 Followup jira to address the client handling issue when chaning from meta replica to non-meta-replica at the server side. (#2768) (#2787)
Signed-off-by: stack <stack@apache.org>
2020-12-18 13:15:25 -08:00
Sandeep Pal
4512163c2f
HBASE-25383: Ability to update and remove peer base config
Closes #2778

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-18 14:21:54 +05:30
shahrs87
47efdbb871
HBASE-25246 Backup/Restore hbase cell tags (#2766)
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-17 12:55:41 +05:30
haxl
98b7aa7765 HBASE-25334 TestRSGroupsFallback.testFallback is flaky (#2775)
Signed-off-by: stack <stack@apache.org>
2020-12-16 13:23:50 -08:00
stack
5eefe62e29 HBASE-25400 [Flakey Tests] branch-2 TestRegionMoveAndAbandon 2020-12-16 10:36:18 -08:00
Baiqiang Zhao
722cbe4294 HBASE-25365 The log in move_servers_rsgroup is incorrect (#2743)
Signed-off-by: stack <stack@apache.org>
2020-12-16 09:15:50 -08:00
Michael Stack
bdc864fb69 HBASE-25389 [Flakey Tests] branch-2 TestMetaShutdownHandler (#2773)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-12-15 09:51:19 -08:00
Adam
cdfacc98dc HBASE-25370 Fix flaky test TestClassFinder#testClassFinderDefaultsToOwnPackage (#2740)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-11 08:24:12 +08:00
Pankaj
ec8eb653f6 HBASE-25277 postScannerFilterRow impacts Scan performance a lot in HBase 2.x (#2675)
* HBASE-25277 postScannerFilterRow impacts Scan performance a lot in HBase 2.x

1. Added a check for Object class in RegionCoprocessorHost to avoid wrong initialization of hasCustomPostScannerFilterRow
2. Removed dummy implementation of postScannerFilterRow from AccessController, VisibilityController & ConstraintProcessor (which are not required currently)

Signed-off-by Ramkrishna S Vasudevan <ramkrishna@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fb6e498b32e48aa606ef5427013fd84452cc762f)
2020-12-11 00:04:41 +05:30
haxl
f526ce6814 HBASE-25287 Forgetting to unbuffer streams results in many CLOSE_WAIT sockets when loading files (#2699)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-10 22:21:54 +08:00
lixiaobao
35fb8f184b HBASE-25372 Fix typo in ban-jersey section of the enforcer plugin in pom.xml (#2749)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-12-09 21:43:07 +08:00
shahrs87
98dc4be77a
HBASE-25328 : Add builder method to create Tags
Closes #2707

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-08 18:23:13 +05:30
stack
bac9764f26 HBASE-25353 [Flakey Tests] branch-2 TestShutdownBackupMaster 2020-12-05 14:23:18 -08:00
stack
b1a5567740 HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Addendum; Reset counter so we actually wait in the new loop added by the
above.
2020-12-05 14:01:12 -08:00
Michael Stack
a0f4c28430 HBASE-25361 [Flakey Tests] branch-2 TestMetaRegionLocationCache.testStandByMetaLocations (#2736)
Add a bit of a wait before testing if online replicas match the zk count. It might take a
while for all replicas to come online.

Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-12-05 08:57:11 -08:00
Andrew Purtell
3ff115ebe4 Preparing development version 2.4.1-SNAPSHOT 2020-12-03 09:52:11 -08:00
Andrew Purtell
282ab70012 Preparing hbase release 2.4.0RC1; tagging and updates to CHANGES.md and RELEASENOTES.md rel/2.4.0 2.4.0RC1 2020-12-03 09:51:41 -08:00
Andrew Purtell
f7bc7be1eb Revert "Preparing development version 2.4.1-SNAPSHOT"
This reverts commit 4127a3940a26d53b12e541c274ebce4e404b0638.
2020-12-03 09:37:38 -08:00
Viraj Jasani
75421af689
Revert "HBASE-25246 Backup/Restore hbase cell tags"
This reverts commit 34721c42ec54b996444b7ea71db7955519771532.
2020-12-03 21:18:03 +05:30
Laxman Goswami
9a2994a5a3
HBASE-25230 Embedded zookeeper server not clean up the old data
Closes #2732

Signed-off-by: maoling <maoling199210191@sina.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-03 14:46:25 +05:30
Andrew Purtell
4127a3940a Preparing development version 2.4.1-SNAPSHOT 2020-12-02 14:04:53 -08:00
Andrew Purtell
108c2ecb52 Preparing hbase release 2.4.0RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 2.4.0RC0 2020-12-02 14:04:49 -08:00
Andrew Purtell
048cee6e47 Rebase RELEASENOTES.md on rel/2.3.0 to bring up to date 2020-12-02 12:53:35 -08:00
Andrew Purtell
ffeed7c664 Refer to earlier releases in CHANGES.md with hyperlinks 2020-12-02 11:13:28 -08:00
Michael Stack
9cc2f76248
HBASE-25349 [Flakey Tests] branch-2 TestRefreshRecoveredReplication.testReplicationRefreshSource:141 Waiting timed out after [60,000] msec (#2731)
Start the check for recovered queue presence earlier.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-12-02 09:55:24 -08:00
shahrs87
34721c42ec
HBASE-25246 Backup/Restore hbase cell tags
Closes #2706

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-12-02 21:30:46 +05:30
Duo Zhang
90e96d1a6e HBASE-25342 Upgrade error prone to 2.4.0 (#2725)
Have to disable MutablePublicArray because of a bug in error prone
https://github.com/google/error-prone/issues/1645

Signed-off-by: stack <stack@apache.org>
2020-12-02 23:38:05 +08:00
stack
745a3a9ab7 Revert "Revert "HBASE-25320 Upgrade hbase-thirdparty dependency to 3.4.1 (#2693)""
This reverts commit 78f30ff496c94f90ced6328c632272dce3582ab4.

This is a revert of a revert; i.e. a repplication -- original patch was
good.
2020-12-01 17:35:43 -08:00