Commit Graph

17277 Commits

Author SHA1 Message Date
stack d75a7001b1 HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionServers
Master force-closes unknown/incorrect Regions OPEN on RS

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Added a note and small refactor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Fix an NPE when CJ ran.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Minor clean up of log message; make it clearer.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 Make it so closeRegionSilentlyAndWait can be used w/o timeout.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 If a RegionServer Report notes a Region is OPEN and the Master does not
 know of said Region, close it (We used to crash out the RegionServer)

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateNode.java
 Minor tweak of toString -- label should be state, not rit (confusing).

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
 Doc.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
 Add region name to exception.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Be more careful about which Regions we queue up for reassign. This
 procedure is run by the operator so could happen at any time. We
 will likely be running this when Master has some accounting of
 cluster members so check its answers for what Regions were on
 server before running.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Doc and we were misrepresenting the case where a Region as not in RIT
 when we got CLOSE -- we were reporting it as though it was already
 trying to CLOSE.
2020-01-03 10:25:18 -08:00
Duo Zhang 85ecdd10a9
HBASE-23587 The FSYNC_WAL flag does not work on branch-2.x (#974)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-01-03 16:57:49 +08:00
Yiran Wu ccfbdadb0f HBASE-23098 [bulkload] If one of the peers in a cluster is configured with NAMESPACE level, its hfile-refs(zk) will be backlogged (#676)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 Signed-off-by: stack <stack@apache.org>
2020-01-02 21:10:41 -08:00
Michael Stack abcb1ee81b HBASE-23632 DeadServer cleanup (#979)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-02 20:25:18 -08:00
stack db7fb063d1 HBASE-23596 HBCKServerCrashProcedure can double assign
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Change its behavior so it will only look in hbase:meta
if the call to the super class turns up zero references.
Only then will it search hbase:meta for references to
'Unknown Servers'. Normal operation where we read Master
context is usual and sufficient. The scan of hbase:meta
is only for case where Master state has been corrupted
and we need to clear out 'Unknown Servers'.
2020-01-02 14:33:54 -08:00
belugabehr 280b94434c HBASE-23628: Remove Apache Commons Digest Base64 (#977)
Signed-off-by: stack <stack@apache.org>
2020-01-02 12:06:19 -08:00
Viraj Jasani 275cdc6052
HBASE-23588 : Cache index & bloom blocks on write if CacheCompactedBlocksOnWrite is enabled
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: chenxu14 <47170471+chenxu14@users.noreply.github.com>
2020-01-02 20:33:59 +05:30
Manu Manjunath 2a4bd0574b HBASE-23629: Add to 'Supporting Projects' in site (#976)
* Update supportingprojects.xml


Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-02 13:38:58 +05:30
Duo Zhang 4cb952ce34
HBASE-23624 Add a tool to dump the procedure info in HFile (#975)
Signed-off-by: stack <stack@apache.org>
2020-01-02 11:23:55 +08:00
Jan Hentschel e8d2931bc1
HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-02 00:39:54 +01:00
Jan Hentschel d86778f4ff
HBASE-23627 Resolved remaining Checkstyle violations in hbase-thrift
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-01-01 22:41:13 +01:00
Jan Hentschel e0eb98c963
HBASE-23625 Reduced number of Checkstyle violations in hbase-common
Signed-off-by: stack <stack@apache.org>
2020-01-01 22:32:37 +01:00
Viraj Jasani e32dbe8ed2
HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRefCount for auto region recovery based on old reader references
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-01 22:16:54 +05:30
binlijin 06eff551c3
HBASE-23615 Use a dedicated thread for executing WorkerMonitor in Pro… (#961)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: virajjasani <34790606+virajjasani@users.noreply.github.com>
2019-12-31 10:03:01 +08:00
Jan Hentschel 923ba7763e
HBASE-23626 Reduced number of Checkstyle violations in tests in hbase-common
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-29 22:39:09 +01:00
Duo Zhang c6a9a4d410 HBASE-23618 Addendum add main method 2019-12-29 22:12:31 +08:00
Duo Zhang bf775628dd
HBASE-23618 Add a tool to dump procedure info in the WAL file (#969)
Signed-off-by: stack <stack@apache.org>
2019-12-29 21:43:50 +08:00
Jan Hentschel 2ca1e46d74
HBASE-23622 Reduced the number of Checkstyle violations in hbase-common
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-28 20:03:58 +01:00
Jan Hentschel 703ee60c90
HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-28 23:03:05 +05:30
Duo Zhang 0ba84d8e95
HBASE-23617 Add a stress test tool for region based procedure store (#962)
Signed-off-by: stack <stack@apache.org>
2019-12-27 22:28:12 +08:00
Viraj Jasani 684a68e945
Adding developer details to pom.xml 2019-12-27 19:36:03 +05:30
Jan Hentschel e9b998012f
HBASE-23619 Used built-in formatting for logger in hbase-zookeeper
Signed-off-by: stack <stack@apache.org>
2019-12-27 10:39:46 +01:00
Guanghao Zhang def9ac7c45
HBASE-23286 Improve MTTR: Split WAL to HFile (#820)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-27 15:59:23 +08:00
Bharath Vissapragada 94346d8623 HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFilter (addendum) (#963)
Having it as static means the test cannot be parameterized (ran into
this issue in HBASE-23305). That happens because the field is not
reset between parameterized runs.
2019-12-26 11:10:39 -08:00
chenxu14 56f9db98d1 HBASE-23374 ExclusiveMemHFileBlock’s allocator should not be hardcoded as ByteBuffAllocator.HEAP
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2019-12-25 21:38:02 +01:00
Duo Zhang 1b049a2d34
HBASE-23326 Implement a ProcedureStore which stores procedures in a HRegion (#941)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-12-25 12:02:12 +08:00
binlijin 9ff2507303
HBASE-23613 ProcedureExecutor check StuckWorkers blocked by DeadServe… (#960)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-25 09:56:48 +08:00
BukrosSzabolcs ee19008b12 HBASE-23589: FlushDescriptor contains non-matching family/output combinations (#949)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-12-24 19:37:06 +08:00
Toshihiro Suzuki 00bb0f20d4 HBASE-23581 Creating table gets stuck when specifying an invalid split policy as METADATA (#942)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Xu Cang <xucang@apache.org>
2019-12-24 19:16:31 +08:00
ravowlga123 fc15ea7546 HBASE-23320 Upgrade surefire plugin to 3.0.0-M4
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-12-23 14:45:38 +01:00
stack 4b6ce0fcbf HBASE-23572 In 'HBCK Report', distringush between live, dead, and unknown servers
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-20 10:54:28 -08:00
binlijin ab40b9648b
HBASE-23564 RegionStates may has some expired serverinfo and make regions do not balance. (#930)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-12-20 15:00:27 +08:00
Duo Zhang f4860d8d2f
HBASE-23594 Procedure stuck due to region happen to recorded on two servers. (#953)
Signed-off-by: stack <stack@apache.org>
2019-12-20 13:40:39 +08:00
XinSun 270eb9886e HBASE-23376 NPE happens while replica region is moving (#906)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-19 15:11:04 +08:00
Duo Zhang 80ba354e2e
HBASE-20461 Implement fsync for AsyncFSWAL (#947)
Signed-off-by: stack <stack@apache.org>
2019-12-19 13:30:15 +08:00
Sean Busbey 17e180e4ee HBASE-23549 Document steps to disable MOB for a column family (#928)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-12-16 12:48:27 -06:00
Sean Busbey bc2f162749 HBASE-23239 Reporting on status of backing MOB files from client-facing cells (#785)
* Adds a new MapReduce job that builds a report of health of mob files
* Also builds background information on mob system use
* add a basic mob architecture in the ref guide to explain how mob takes the reference hfile value and finds the actual cell contents
* add a troubleshooting mob section to the ref guide to explain how to do a mob reference scan.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-12-16 12:48:17 -06:00
Bharath Vissapragada b38446c017 HBASE-23575 Remove dead code in AsyncRegistry (addendum)
Additions to MiniHBaseCluster needed for branch-2.

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-12-16 10:32:44 -06:00
Ankit Singhal 29d1a97482 HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-12-16 23:44:05 +09:00
ramkrish86 eee9480cb4
HBASE-23066 Allow cache on write during compactions when prefetching … (#935)
* HBASE-23066 Allow cache on write during compactions when prefetching is enabled

* Fix checkstyle issues - recommit
2019-12-16 15:20:30 +05:30
Junegunn Choi ec317a6629 HBASE-23582 Unbalanced braces in string representation of table descriptor
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2019-12-16 10:35:14 +01:00
Beata Sudi de87d72bb4 HBASE-22920 github pr testing job should use dev-support script (#883)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2019-12-13 15:56:20 -08:00
David Mollitor 3be8ae2167 HBASE-23380 General cleanup of FSUtil (#912)
* Clean up JavaDocs
* Clean up logging and error messages
* Remove superfluous code
* Replace static code with library call
* Do not swallow Interrupted Exceptions
* Use try-with-resources
* User multi-Exception catches to reduce code size

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-13 16:32:18 -06:00
Bharath Vissapragada efa4fe901a HBASE-23575 Remove dead code in AsyncRegistry (#929)
Removes a bunch of dead code and fixes some checkstyle nits.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-13 13:26:03 -06:00
Michael Stack 2d76457577
HBASE-23570 Point users to the async-profiler home page if diagrams are coming up blank (#937) 2019-12-12 16:51:04 -08:00
BukrosSzabolcs 413d4b2d0f HBASE-23566: Fix package/packet terminology problem in chaos monkeys (#933)
s/package/packet/g

Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-12 16:32:26 -06:00
meiyi 85a081925b HBASE-23555 TestQuotaThrottle is broken (#924) 2019-12-11 12:23:13 -08:00
stack b0233a294f HBASE-23554 Encoded regionname to regionname utility (#923); ADDENDUM 2019-12-11 11:02:59 -08:00
Michael Stack 8742265d85 HBASE-23554 Encoded regionname to regionname utility (#923)
Adds shell command regioninfo:

  hbase(main):001:0>  regioninfo '0e6aa5c19ae2b2627649dc7708ce27d0'
  {ENCODED => 0e6aa5c19ae2b2627649dc7708ce27d0, NAME => 'TestTable,,1575941375972.0e6aa5c19ae2b2627649dc7708ce27d0.', STARTKEY => '', ENDKEY => '00000000000000000000299441'}
  Took 0.4737 seconds

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-11 08:57:59 -08:00
Sean Busbey ec9bd20ec8 Revert "HBASE-23066 Allow cache on write during compactions when prefetching is (#919)"
TestCacheOnWrite failing all the time.

This reverts commit d561130e82.
2019-12-11 10:17:46 -06:00