Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,694 Commits 95 Branches 866 Tags
Commit Graph

18694 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Duo Zhang
04a4ac5ae9 HBASE-27728 Implement a tool to migrate replication peer data between different storage implementation (#5179) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 31c4aea48c3d4bc7585784dd65375edbb4e61a8a)

Conflicts:
	hbase-client/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeerConfig.java
	hbase-replication/src/test/java/org/apache/hadoop/hbase/replication/ReplicationPeerStorageTestBase.java
 2023-04-17 22:47:52 +08:00
Ray Mattingly
1ae057b89f
HBASE-27536: Include more request information in slowlog for Scans (#5166) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2023-04-16 10:11:35 -04:00
Kota-SH
b75c72d925 HBASE-27422: Support replication for hbase:acl (#4827) 
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

(cherry picked from commit 856fa145372b85e7f600f7c959b69ee62702dfb1)
 2023-04-14 10:27:19 +01:00
Duo Zhang
3e6e9c5cca HBASE-27727 Implement filesystem based Replication peer storage (#5165) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit a71105997f2c08c74f3713925847cce3d44134bc)
 2023-04-13 19:45:20 +08:00
Bryan Beaudreault
08489365a5
HBASE-27768 Race conditions in BlockingRpcConnection (#5154) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2023-04-10 15:23:03 -04:00
Bryan Beaudreault
a67a8f7fd3 HBASE-27780 FileChangeWatcher improvements (#5164) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-04-08 10:25:32 -04:00
tianhang
6305c7a4f2 HBASE-27713 Remove numRegions in Region Metrics (#5107) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 251e4984776e2ee13ae4df1de4c000f35a58ad59)
 2023-04-08 15:09:25 +08:00
Nick Dimiduk
f8c234e8d0 HBASE-27720 TestClusterRestartFailover is flakey due to metrics assertion (#5110) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f36587c6486468f6b19f7419c9a263ed81d8bdc7)
 2023-04-08 11:05:33 +08:00
chenglei
62ea717b31
HBASE-27778 Incorrect ReplicationSourceWALReader. totalBufferUsed may cause replication hang up (#5160) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-04-06 21:37:10 +08:00
bsglz
85793655d3 HBASE-27765 Add biggest cell related info into web ui (#5151) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit a363b76d815a71c4acd96abca1a2578af57ede78)
 2023-04-05 23:18:13 +08:00
Bryan Beaudreault
cc00b8b8a8 HBASE-27758 Inconsistent synchronization in MetricsUserSourceImpl (#5149) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-03-31 09:17:30 -04:00
Bryan Beaudreault
9f4b31e619
HBASE-27704 Quotas can drastically overflow configured limit (#5153) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2023-03-30 09:13:00 -04:00
Bryan Beaudreault
e77e3ffa8f
HBASE-27756 Make request guardrail configs in RSRpcServices live updatable (#5152) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2023-03-30 09:12:26 -04:00
Rishabh Murarka
753eaefa29 HBASE-27726 Handling of ruby shell SyntaxError exceptions (#5147) 
Co-authored-by: Rishabh Murarka <rmurarka@visa.com>
 2023-03-29 14:23:15 +05:30
Xiaolin Ha
45e3451997
HBASE-27333 Abort RS when the hostname is different from master seen (#4732) (#5148) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2023-03-29 11:42:32 +08:00
Bryan Beaudreault
e33afb4524
HBASE-27532 Add block bytes scanned metrics (#5146) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2023-03-28 13:56:37 -04:00
Kota-SH
61e18a3d6c HBASE-27750: Update the list of prefetched Hfiles upon block eviction (#5140) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2023-03-27 10:57:57 +01:00
dependabot[bot]
6d9ac13f80 HBASE-27748 Bump jettison from 1.5.2 to 1.5.4 (#5136) 
Bumps [jettison](https://github.com/jettison-json/jettison) from 1.5.2 to 1.5.4.
- [Release notes](https://github.com/jettison-json/jettison/releases)
- [Commits](https://github.com/jettison-json/jettison/compare/jettison-1.5.2...jettison-1.5.4)

---
updated-dependencies:
- dependency-name: org.codehaus.jettison:jettison
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 735fb4338876b8d7da9dcad7383bdf1fc9c90c87)
 2023-03-24 21:56:49 +08:00
Duo Zhang
17d15a135a HBASE-27732 NPE in TestBasicWALEntryStreamFSHLog.testEOFExceptionInOldWALsDirectory (#5119) 
Add a 'closed' flag in WALProps in AbstractFSWAL to indicate that whether a WAL
file has been closed, if not, we will not try to archive it. Will mark it as
closed after we fully close it in the background close task, and try to archive
again.

Also modified some tests since now the archiving of a rolled WAL file is also
asynchronous, we need to wait instead of asserting directly.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 230fdc0b50578144d3e0916c33d4532bb8644763)
 2023-03-24 17:43:30 +08:00
Duo Zhang
3b24b3b969 HBASE-26866 Shutdown WAL may abort region server (#4254) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit b67c16a7636958970d37bfcd775fd55e8de98177)
 2023-03-24 15:59:29 +08:00
Xiaolin Ha
62fc8c81a3 HBASE-26526 Introduce a timeout to shutdown of WAL (#3297) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit ca3ba494cbc322b0824d2d755bcf4191c3a525ed)
 2023-03-24 15:58:49 +08:00
Andrew Purtell
dd3cdf13cf HBASE-27744 Update compression dependencies (#5137) 
Update compression codec dependencies:

 - aircompressor 0.21 -> 0.24
 - brotli4j 1.8.0 -> 1.11.0
 - snappy 1.1.8.4 -> 1.1.9.1
 - zstd-jni 1.5.0-4 -> 1.5.4-2

and add a new supplemental model for new brotli4j dependency.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-03-23 13:10:28 -07:00
Nick Dimiduk
29ca26a765
HBASE-27741 Fall back to protoc osx-x86_64 on Apple Silicon 
Old protobuf-2.5 doesn't know about Apple Silicon. Fall back to the x86 binary and hope that
rosetta continues to work.

https://cwiki.apache.org/confluence/display/HADOOP/Develop+on+Apple+Silicon+%28M1%29+macOS

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2023-03-22 14:24:07 +01:00
Xiaolin Ha
d4edfe404e HBASE-27676 Scan handlers in the RPC executor should match at least one scan queues (#5074) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2023-03-22 08:10:56 +08:00
xuzifu666
a10cbf8319 HBASE-27736 HFileSystem.getLocalFs is not used (#5125) 
Co-authored-by: xuyu <11161569@vivo.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit dd3b1630e848d34db432526dee86289439594510)
 2023-03-21 22:55:16 +08:00
Nick Dimiduk
29dfd005cf HBASE-27737 Add supplemental model for com.aayushatharva.brotli4j:native-osx-aarch64 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2023-03-21 15:23:43 +01:00
Victor
ff9754049b
HBASE-27684: add client metrics related to user region lock. (#5081) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Rushabh Shah <shahrs87@apache.org>
Signed-off-by: Tanuj Khurana <tkhurana@apache.org>
 2023-03-20 22:36:08 -07:00
Nick Dimiduk
e7a958a45f HBASE-27651 hbase-daemon.sh foreground_start should propagate SIGHUP and SIGTERM 
Introduce separate `trap`s for SIGHUP vs. the rest. Treat `SIGINT`, `SIGKILL`, and `EXIT` identically, as before. Use the signal name without `SIG` prefix for increased portability, as per the POSIX man page for `trap`.

`SIGTERM` handler will now honor `HBASE_STOP_TIMEOUT` as described in the file header.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2023-03-20 15:36:33 +01:00
Xiaolin Ha
873f8987f1
HBASE-27646 Should not use pread when prefetching in HFilePreadReader (#5063) (#5122) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2023-03-20 19:26:17 +08:00
Duo Zhang
81dffda0d3 HBASE-27702 Remove 'hbase.regionserver.hlog.writer.impl' config (#5096) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit 997d1326013ff2183d7ebcc5d0e4729f118362ca)
 2023-03-20 11:05:02 +08:00
chaijunjie0101
a6b99faa00 HBASE-27718 The regionStateNode only need remove once in regionOffline (#5106) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit e6977a95975580831a31d122c800d01cadac007a)
 2023-03-19 20:19:37 +08:00
slfan1989
8f6d6b9ec3 HBASE-27729. Missed one parameter when logging exception in StoreFileListFile. (#5114) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 3c2c685892740af68428f8bb715ca4234b6ff320)
 2023-03-19 20:05:44 +08:00
Nihal Jain
b1a00be1bc
HBASE-27671 Client should not be able to restore/clone a snapshot after it has TTL expired it's TTL has expired (#5117) ( #5109) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2023-03-18 00:07:16 -07:00
Bryan Beaudreault
5de1e4f88e
HBASE-27710 ByteBuff ref counting is too expensive for on-heap buffers (#5115) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-03-17 15:24:28 -04:00
Kota-SH
d136c6d7c5 HBASE-27686: Recovery of BucketCache and Prefetched data after RS Crash (#5080) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 58cb1f4799e90f5fca51f76aa4a9787d823c69a5)
 2023-03-16 15:44:59 +00:00
Nick Dimiduk
59434e9cd4 HBASE-27708 CPU hot-spot resolving User subject 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-03-16 13:57:02 +01:00
Nick Dimiduk
dbb78388e5
HBASE-27652 Client-side lock contention around Configuration when using read replica regions (#5036) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-03-15 16:07:48 +01:00
Duo Zhang
ef8a981f22 HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntryStream (#5105) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit 1f2e1f5b3a18504b3192f3651397af7f90b04932)
 2023-03-15 21:49:45 +08:00
Duo Zhang
b5dcd48c64 HBASE-27716 Fix TestWALOpenAfterDNRollingStart (#5102) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit c8bee238fc51b82215cea844030a7a4dc9f6499c)
 2023-03-15 11:03:59 +08:00
Duo Zhang
e051400a6d HBASE-27714 WALEntryStreamTestBase creates a new HBTU in startCluster method which causes all sub classes are testing default configurations (#5101) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit ce7b1d4e4ff2b5456c8655558378a127f3274107)
 2023-03-15 10:53:46 +08:00
alan.zhao
6920e72f50
HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0 (#5097) 
Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-03-15 08:17:18 +08:00
tianhang
24653dd105
HBASE-27615 Add group of regionServer on Master webUI (#5098) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2023-03-14 11:55:18 +08:00
Horváth Dóra
0f67503b9d HBASE-27685 Enable code coverage reporting to SonarQube in HBase (#5076) 
(cherry picked from commit 72d5a46899ef0866d777f0a9508649beed6d8dd3)
 2023-03-10 20:16:08 +01:00
Duo Zhang
cec0556263 HBASE-27632 Refactor WAL.Reader implementation so we can better support WAL splitting and replication (#5055) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit e48c4485db8d9255510ee1cc9cf465e14de637d7)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplaySyncReplicationWALCallable.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogTestHelper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestSyncReplicationActive.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALReaderOnSecureWAL.java
 2023-03-10 23:03:17 +08:00
Rajeshbabu Chintaguntla
e180aa10df HBASE-27669 chaos-daemon.sh should make use hbase script start/stop chaosagent and chaos monkey runner (#5078) 
Co-authored-by: Rajeshbabu Chintaguntla <rchintag@visa.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit bc8b13e468a258b4ee47e40cf3645bf27d66471d)
 2023-03-09 10:49:42 +08:00
tianhang
43fc4bfcd8
HBASE-27681 Refactor Table Latency Metrics (#5093) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2023-03-08 10:13:14 -05:00
huaxiangsun
9a69a69648 HBASE-27250 MasterRpcService#setRegionStateInMeta does not support replica region encodedNames or region names 
- Added sanity check to make sure input region encoded name or region name is valid
- Assignment improvements pertaining to read replica regions
- make several MetaTableAccessor methods more precise in their handling of replica regions
- hbck2 setRegionStateInMeta and HBCKServerCrashProcedure handle read replicas
- separate AM helper methods -- loading RegionInfo from cache vs. refreshing cache from meta
- AM helper method support loading RegionInfo from cache via either region name and encoded region
  name (both caches are maintained, and under lock)
- consolidate, extend tests to cover read replica regions

Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2023-03-06 11:02:12 +01:00
Xiaolin Ha
1ac6f1689f
HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#4536) (#5085) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2023-03-06 14:21:22 +08:00
Xiaolin Ha
505a901fed
HBASE-27672 Read RPC threads may BLOCKED at the Configuration.get when using java compression (#5075) (#5083) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2023-03-06 12:53:30 +08:00
alan.zhao
f1e970d3d4 HBASE-27670 Improve FSUtils to directly obtain FSDataOutputStream (#5064) 
Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 59fdaa28f726dcd65fbfb58f9ea6aa499f2a903c)
 2023-03-04 21:58:59 +08:00