48fda91c5b
HBASE-24380 : Provide WAL splitting journal logging ( #1860 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredEditsOutputSink.java
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
2020-06-19 16:27:48 -07:00
7b171a3c1a
HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING ( #1912 )
...
Addendum; minor log edits
2020-06-18 08:37:19 -07:00
6eb93287cc
HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING ( #1912 )
...
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
Don't register as a chore on construction if no coordination state
manager instance (there is no instance when procv2 WAL splitter).
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Edit logs.
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALRemoteProcedure.java
Add proc name rather than rely on default behavior. Add detail to the
toString.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Factoring
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
Print the maxLogs... we don't do it any where.
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
Utility method to strip prefix from wal path.
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: tianjingyun <tianjy@apache.org>
2020-06-18 08:32:48 -07:00
ef809c198b
HBASE-24205 - Create metric to know the number of reads that happens ( #1920 )
...
* HBASE-24205 - Create metric to know the number of reads that happens
from memstore (branch-2)
* Add the optimization as in master and fix whitestyle and checkstyle
* Fix compilation error that accidently crept in
Authored-by: Ramkrishna <ramkrishna@apache.org>
Signed-off by:Anoop Sam John<anoopsamjohn@gmail.com>
Signed-off by:Viraj Jasani<virajjasani@apache.org>
2020-06-18 18:59:46 +05:30
812d1e2bb5
HBASE-24577 Doc WALSplitter classes ( #1913 )
...
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-17 14:59:30 -07:00
e202c98fb0
HBASE-24518 : waitForNamespaceOnline() should return false if any region is offline ( #1869 )
...
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-06-17 20:05:15 +05:30
c49df36e85
HBASE-24564: Make RS abort call idempotent.
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 2567d15218
2020-06-16 08:43:06 -07:00
36ee96ca36
HBASE-24478 The regionInfo parameter for MasterProcedureScheduler#wai… ( #1819 )
...
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: clarax <clarax98007@gmail.com>
2020-06-16 17:35:17 +08:00
cb3d0d7d21
HBASE-24015: Test for Assign and Unassign of Regions on RegionServer on failure ( #1898 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-16 12:54:02 +05:30
035ea516e9
HBASE-24038 Add a metric to show the locality of ssd in table.jsp ( #1902 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-15 19:28:07 +05:30
700c5a7e3b
HBASE-24517 Addendum add more comments ( #1882 )
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-15 10:44:30 +08:00
6ff6fe49a3
HBASE-8458 Support for batch version of checkAndMutate() ( #1897 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-14 13:55:06 +09:00
2d57595f54
HBASE-23195 FSDataInputStreamWrapper unbuffer can NOT invoke the classes that NOT implements CanUnbuffer but its parents class implements CanUnbuffer
...
Closes #746
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-12 13:27:53 -04:00
a4e1d073f4
HBASE-24545 Add backoff to SCP check on WAL split completion ( #1891 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-12 07:58:02 -07:00
d187ce2029
HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master ( #1885 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-12 16:52:15 +05:30
043a9e862f
HBASE-24529 hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles ( #1881 )
...
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
2020-06-12 19:02:20 +09:00
ed7dc9ed4c
HBASE-24189 WALSplit recreates region dirs for deleted table with recovered edits data.
2020-06-11 21:28:32 +05:30
1b1c269911
HBASE-24532 : Execute region plans with throttle should return succeeded plans ( #1884 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-06-11 18:23:32 +05:30
c02554024f
HBASE-24511 Ability to configure timeout between RPC retry to RS from master ( #1861 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-10 13:04:47 +05:30
dc2cf6ab5c
HBASE-24441 CacheConfig details logged at Store open is not really u… ( #1801 )
...
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-06-09 14:19:15 +08:00
101023bcc1
HBASE-24468 Add region info when log meessages in HStore. ( #1803 )
...
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-09 14:14:05 +08:00
02e7beaca1
HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL ( #1874 ) ( #1875 )
...
Co-authored-by: Guangxu Cheng <guangxucheng@gmail.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2020-06-08 22:51:48 -07:00
c5dacfbbea
HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may cause SCP to accidentally skip assigning a region ( #1865 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2020-06-09 11:16:49 +08:00
6d96694a25
HBASE-24517 AssignmentManager.start should add meta region to ServerStateNode ( #1866 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2020-06-09 08:27:17 +08:00
9ba3be61b3
HBASE-24444 Should shutdown mini cluster after class in TestMetaAssignmentWithStopMaster ( #1802 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-09 08:21:52 +08:00
82c50d72e2
HBASE-24208 Remove RS entry from zk draining servers node after RS been stopped ( #1841 )
...
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-08 22:50:26 +05:30
67fa367bb6
HBASE-24359 Optionally ignore edits for deleted CFs for replication ( #1855 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-08 13:14:50 +08:00
108e23630a
HBASE-24506 async client deadlock ( #1858 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-06-08 09:57:33 +08:00
467efa573c
HBASE-24515 batch Increment/Append fails when retrying the RPC
...
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2020-06-08 10:41:41 +09:00
0c9806faec
HBASE-24496 The tag of Base Stats not actived by default in table.jsp ( #1837 )
...
Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-06 19:17:41 +05:30
72dadcd615
HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange ( #1850 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-05 15:24:39 +08:00
82b8feecef
HBASE-24505 Reimplement Hbck.setRegionStateInMeta ( #1852 )
...
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2020-06-05 08:55:27 +08:00
cc4c28c575
HBASE-24456 : Create ImmutableScan and use it for CustomizedScanInfoBuilder ( #1818 )
...
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-04 16:30:32 +05:30
4a84e28937
Revert "HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange ( #1840 )"
...
This reverts commit 8815215b3e
2020-06-04 12:17:15 +08:00
8815215b3e
HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange ( #1840 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-06-04 11:18:25 +08:00
e59f39e09e
HBASE-21905: [FIFOCompaction] Don't compact already inflight store files ( #1829 )
...
This one surfaced as a flake test but turns out to be a legit bug
in FIFOCompaction code. FifoCompaction does not check if an empty
store file is already being compacted by an in-flight compaction
request and still enqueues. It can potentially race with a running
compaction (as in this test case, see jira for the exact exception).
Fixes the edge case and cleans up the test code a bit.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-06-03 17:03:24 -07:00
4884773676
HBASE-24418 Consolidate Normalizer implementations
...
Simplify our Normalizer story to have just a single, configurable
implementation.
* fold the features of `MergeNormalizer` into
`SimpleRegionNormalizer`, removing the intermediate abstract class.
* configuration keys for merge-only features now share a common
structure.
* add configuration to selectively disable normalizer split/merge
operations.
* `RegionNormalizer` now extends `Configurable` instead of creating a
new instance of `HBaseConfiguration` or snooping one off of other
fields.
* avoid the extra RPCs by using `MasterServices` instead of
`MasterRpcServices`.
* boost test coverage of all the various flags and feature
combinations.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-06-03 16:43:40 -07:00
11d093bc39
HBASE-21406 "status 'replication'" should not show SINK if the cluste… ( #1761 )
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(Cherry picked from commit e5345b3a7c
2020-06-03 09:33:36 +01:00
b3c6af9500
HBASE-24492 : Remove infinite loop from ProtobufLogReader#readNext ( #1831 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-03 13:44:19 +05:30
571ef7e811
HBASE-24412 Canary support check only one column family per RegionTask ( #1817 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-03 09:16:11 +08:00
9c8895cfbf
HBASE-24474 Rename LocalRegion to MasterRegion ( #1811 )
...
Addendum: HRegion changed size.
2020-06-02 10:44:48 -07:00
6b43015fc9
HBASE-24474 Rename LocalRegion to MasterRegion ( #1811 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2020-06-02 10:43:27 +08:00
a7a8afdf6a
HBASE-24479: Deflake TestCompaction#testStopStartCompaction ( #1820 )
...
Polling of active compaction count is racy. Tightened the asserts
to be more reliable.
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-05-31 23:25:39 -07:00
d7f388518c
HBASE-24477: Move ConfigurationObserver and related classes to hbase-common
...
This utility is useful for any module that wants to detect
dynamic config changes. Having it to hbase-common makes it
accessible to all the other modules.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-31 14:14:30 -07:00
327d090e1b
HBASE-24475 Clean up the master thread name getting in SplitLogManager and AssignmentManager ( #1810 )
2020-05-30 22:24:05 -07:00
7b17e1cdaa
HBASE-24454 - Read ioErrorStartTime to local temporary variable to avoid issue when it is set to -1 between greater than zero check and calculation of error duration ( #1816 )
...
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-30 21:07:46 -07:00
4d5efec767
HBASE-24471 The way we bootstrap meta table is confusing ( #1806 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2020-05-30 16:37:41 +08:00
2dd160e4c7
HBASE-23716 MasterFileSystem should not create MasterProcWALs dir on … ( #1078 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
2020-05-30 16:37:30 +08:00
1ac00e11cb
Revert "HBASE-23716 MasterFileSystem should not create MasterProcWALs dir on … ( #1078 )"
...
Wrong jira id
This reverts commit aca7f6fcf3
2020-05-30 16:37:19 +08:00
b2d24a1812
HBASE-23941 : FilterBy operator support in get_slowlog_responses API ( #1793 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-05-30 13:57:32 +05:30
Viraj Jasani
stack
Michael Stack
ramkrish86
Bharath Vissapragada
xincunSong
Sandeep Pal
bsglz
Duo Zhang
Toshihiro Suzuki
zhaoym6
sguggilam
Anoop Sam John
huaxiangsun
WenFeiYi
gkanade
XinSun
Nick Dimiduk
Wellington Ramos Chevreuil
niuyulin
jacob-leblanc