Commit Graph

8943 Commits

Author SHA1 Message Date
Duo Zhang 438d93b30f HBASE-27822 TestFromClientSide5.testAppendWithoutWAL is flaky (#5211)
Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit b59eb96407)
2023-05-04 22:24:56 +08:00
Duo Zhang 835077fa35 HBASE-27821 Split TestFuzzyRowFilterEndToEnd (#5209)
Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit 4e69921a00)
2023-05-04 22:05:42 +08:00
Duo Zhang bf20017148
HBASE-27806 Addendum fix error prone error (#5214) 2023-05-03 19:52:01 +08:00
Nick Dimiduk da81079365 HBASE-27762 Include EventType and ProcedureV2 pid in logging via MDC (#5145)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-05-01 17:45:05 +02:00
Duo Zhang ce4a2bf77f HBASE-27819 10k RpcServer.MAX_REQUEST_SIZE is not enough in ReplicationDroppedTable related tests (#5208)
Also modified the tests so we will only start the cluster once instead
of start it in setUpBeforeClass and then restart it when running the
actual test method

Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit 21d61cfa1c)
2023-05-01 21:22:09 +08:00
Nihal Jain adfcea8073 HBASE-27792 Guard Master/RS Dump Servlet behind admin walls (#5176) 2023-04-28 19:39:03 +05:30
Duo Zhang aa7486d598 HBASE-27818 Split TestReplicationDroppedTables (#5206)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 1ced254b8d)
2023-04-28 10:00:02 +08:00
Nihal Jain 7434374900 HBASE-20804 Document and add tests for HBaseConfTool (#5190)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit e86f930894)
2023-04-23 23:43:35 +08:00
haosen chen 157df067a4 HBASE-27807 PressureAwareCompactionThroughputController#tune log the opposite of the actual scenario (#5198)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 95f83cb8e2)
2023-04-23 23:34:08 +08:00
Duo Zhang fe2992a312 HBASE-27806 Support dynamic reinitializing replication peer storage (#5195)
Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit 18ae733b15)
2023-04-23 23:15:27 +08:00
Ray Mattingly e9ffc1b07e
HBASE-27535: Separate slowlog thresholds for scans vs other requests (#5188)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2023-04-21 21:20:04 -07:00
chenglei 0787199feb
HBASE-27785 Encapsulate and centralize totalBufferUsed in ReplicationSourceManager (#5196)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-04-21 21:52:40 +08:00
Duo Zhang 92a2868f37
HBASE-27783 Implement a shell command to disable all peer modification (#5182)(#5191)
Signed-off-by: Liangjun He <heliangjun@apache.org>
2023-04-21 09:57:28 +08:00
Nihal Jain f6900f8c63 HBASE-27791 Upgrade vega and its related js libraries (#5173)
Signed-off-by: Rajeshbabu Chintaguntla <rajeshbabu@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2023-04-18 12:15:07 +02:00
Ray Mattingly 1ae057b89f
HBASE-27536: Include more request information in slowlog for Scans (#5166)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-04-16 10:11:35 -04:00
Kota-SH b75c72d925 HBASE-27422: Support replication for hbase:acl (#4827)
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

(cherry picked from commit 856fa14537)
2023-04-14 10:27:19 +01:00
Duo Zhang 3e6e9c5cca HBASE-27727 Implement filesystem based Replication peer storage (#5165)
Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit a71105997f)
2023-04-13 19:45:20 +08:00
Nick Dimiduk f8c234e8d0 HBASE-27720 TestClusterRestartFailover is flakey due to metrics assertion (#5110)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f36587c648)
2023-04-08 11:05:33 +08:00
chenglei 62ea717b31
HBASE-27778 Incorrect ReplicationSourceWALReader. totalBufferUsed may cause replication hang up (#5160)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-04-06 21:37:10 +08:00
bsglz 85793655d3 HBASE-27765 Add biggest cell related info into web ui (#5151)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit a363b76d81)
2023-04-05 23:18:13 +08:00
Bryan Beaudreault 9f4b31e619
HBASE-27704 Quotas can drastically overflow configured limit (#5153)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2023-03-30 09:13:00 -04:00
Bryan Beaudreault e77e3ffa8f
HBASE-27756 Make request guardrail configs in RSRpcServices live updatable (#5152)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2023-03-30 09:12:26 -04:00
Xiaolin Ha 45e3451997
HBASE-27333 Abort RS when the hostname is different from master seen (#4732) (#5148)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2023-03-29 11:42:32 +08:00
Bryan Beaudreault e33afb4524
HBASE-27532 Add block bytes scanned metrics (#5146)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2023-03-28 13:56:37 -04:00
Kota-SH 61e18a3d6c HBASE-27750: Update the list of prefetched Hfiles upon block eviction (#5140)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2023-03-27 10:57:57 +01:00
Duo Zhang 17d15a135a HBASE-27732 NPE in TestBasicWALEntryStreamFSHLog.testEOFExceptionInOldWALsDirectory (#5119)
Add a 'closed' flag in WALProps in AbstractFSWAL to indicate that whether a WAL
file has been closed, if not, we will not try to archive it. Will mark it as
closed after we fully close it in the background close task, and try to archive
again.

Also modified some tests since now the archiving of a rolled WAL file is also
asynchronous, we need to wait instead of asserting directly.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 230fdc0b50)
2023-03-24 17:43:30 +08:00
Duo Zhang 3b24b3b969 HBASE-26866 Shutdown WAL may abort region server (#4254)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit b67c16a763)
2023-03-24 15:59:29 +08:00
Xiaolin Ha 62fc8c81a3 HBASE-26526 Introduce a timeout to shutdown of WAL (#3297)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit ca3ba494cb)
2023-03-24 15:58:49 +08:00
Xiaolin Ha d4edfe404e HBASE-27676 Scan handlers in the RPC executor should match at least one scan queues (#5074)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-03-22 08:10:56 +08:00
xuzifu666 a10cbf8319 HBASE-27736 HFileSystem.getLocalFs is not used (#5125)
Co-authored-by: xuyu <11161569@vivo.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit dd3b1630e8)
2023-03-21 22:55:16 +08:00
Victor ff9754049b
HBASE-27684: add client metrics related to user region lock. (#5081)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Rushabh Shah <shahrs87@apache.org>
Signed-off-by: Tanuj Khurana <tkhurana@apache.org>
2023-03-20 22:36:08 -07:00
Xiaolin Ha 873f8987f1
HBASE-27646 Should not use pread when prefetching in HFilePreadReader (#5063) (#5122)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-03-20 19:26:17 +08:00
Duo Zhang 81dffda0d3 HBASE-27702 Remove 'hbase.regionserver.hlog.writer.impl' config (#5096)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit 997d132601)
2023-03-20 11:05:02 +08:00
chaijunjie0101 a6b99faa00 HBASE-27718 The regionStateNode only need remove once in regionOffline (#5106)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit e6977a9597)
2023-03-19 20:19:37 +08:00
slfan1989 8f6d6b9ec3 HBASE-27729. Missed one parameter when logging exception in StoreFileListFile. (#5114)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 3c2c685892)
2023-03-19 20:05:44 +08:00
Nihal Jain b1a00be1bc
HBASE-27671 Client should not be able to restore/clone a snapshot after it has TTL expired it's TTL has expired (#5117) ( #5109)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2023-03-18 00:07:16 -07:00
Kota-SH d136c6d7c5 HBASE-27686: Recovery of BucketCache and Prefetched data after RS Crash (#5080)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 58cb1f4799)
2023-03-16 15:44:59 +00:00
Nick Dimiduk dbb78388e5
HBASE-27652 Client-side lock contention around Configuration when using read replica regions (#5036)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-03-15 16:07:48 +01:00
Duo Zhang ef8a981f22 HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntryStream (#5105)
Signed-off-by: Liangjun He <heliangjun@apache.org>
(cherry picked from commit 1f2e1f5b3a)
2023-03-15 21:49:45 +08:00
Duo Zhang b5dcd48c64 HBASE-27716 Fix TestWALOpenAfterDNRollingStart (#5102)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit c8bee238fc)
2023-03-15 11:03:59 +08:00
Duo Zhang e051400a6d HBASE-27714 WALEntryStreamTestBase creates a new HBTU in startCluster method which causes all sub classes are testing default configurations (#5101)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit ce7b1d4e4f)
2023-03-15 10:53:46 +08:00
alan.zhao 6920e72f50
HBASE-27688 HFile splitting occurs during bulkload, the CREATE_TIME_TS of hfileinfo is 0 (#5097)
Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-03-15 08:17:18 +08:00
tianhang 24653dd105
HBASE-27615 Add group of regionServer on Master webUI (#5098)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2023-03-14 11:55:18 +08:00
Duo Zhang cec0556263 HBASE-27632 Refactor WAL.Reader implementation so we can better support WAL splitting and replication (#5055)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit e48c4485db)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplaySyncReplicationWALCallable.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogTestHelper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestSyncReplicationActive.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALReaderOnSecureWAL.java
2023-03-10 23:03:17 +08:00
tianhang 43fc4bfcd8
HBASE-27681 Refactor Table Latency Metrics (#5093)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-03-08 10:13:14 -05:00
huaxiangsun 9a69a69648 HBASE-27250 MasterRpcService#setRegionStateInMeta does not support replica region encodedNames or region names
- Added sanity check to make sure input region encoded name or region name is valid
- Assignment improvements pertaining to read replica regions
- make several MetaTableAccessor methods more precise in their handling of replica regions
- hbck2 setRegionStateInMeta and HBCKServerCrashProcedure handle read replicas
- separate AM helper methods -- loading RegionInfo from cache vs. refreshing cache from meta
- AM helper method support loading RegionInfo from cache via either region name and encoded region
  name (both caches are maintained, and under lock)
- consolidate, extend tests to cover read replica regions

Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2023-03-06 11:02:12 +01:00
Xiaolin Ha 1ac6f1689f
HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#4536) (#5085)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2023-03-06 14:21:22 +08:00
alan.zhao f1e970d3d4 HBASE-27670 Improve FSUtils to directly obtain FSDataOutputStream (#5064)
Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 59fdaa28f7)
2023-03-04 21:58:59 +08:00
Jan Hentschel 6bc84de563 HBASE-23983 Fixed Spotbugs complaint in RegionStates related to ignored return value
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-03-03 10:40:03 +01:00
Xiaolin Ha 0c85952a8f
HBASE-27458 Use ReadWriteLock for region scanner readpoint map (#4859) (#5073)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2023-03-02 13:49:02 +08:00