Commit Graph

17093 Commits

Author SHA1 Message Date
Nick Dimiduk f98085e44e HBASE-23333 Include Call.toShortString() in sendCall exceptions 2020-01-03 12:23:37 -08:00
stack c44a5c47dd HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionServers
Master force-closes unknown/incorrect Regions OPEN on RS

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Added a note and small refactor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Fix an NPE when CJ ran.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Minor clean up of log message; make it clearer.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 Make it so closeRegionSilentlyAndWait can be used w/o timeout.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 If a RegionServer Report notes a Region is OPEN and the Master does not
 know of said Region, close it (We used to crash out the RegionServer)

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateNode.java
 Minor tweak of toString -- label should be state, not rit (confusing).

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
 Doc.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
 Add region name to exception.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Be more careful about which Regions we queue up for reassign. This
 procedure is run by the operator so could happen at any time. We
 will likely be running this when Master has some accounting of
 cluster members so check its answers for what Regions were on
 server before running.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Doc and we were misrepresenting the case where a Region as not in RIT
 when we got CLOSE -- we were reporting it as though it was already
 trying to CLOSE.
2020-01-03 10:21:55 -08:00
Duo Zhang 26b1695df5 HBASE-23587 The FSYNC_WAL flag does not work on branch-2.x (#974)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-01-03 17:01:40 +08:00
Guanghao Zhang a693a8fd95 HBASE-23286 Improve MTTR: Split WAL to HFile (#820)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-03 16:39:06 +08:00
Yiran Wu 381ff85e3d HBASE-23098 [bulkload] If one of the peers in a cluster is configured with NAMESPACE level, its hfile-refs(zk) will be backlogged (#676)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 Signed-off-by: stack <stack@apache.org>
2020-01-02 21:13:08 -08:00
Michael Stack 8cf5d045e0 HBASE-23632 DeadServer cleanup (#979)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-02 20:23:52 -08:00
stack 1b366719a0 HBASE-23596 HBCKServerCrashProcedure can double assign
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Change its behavior so it will only look in hbase:meta
if the call to the super class turns up zero references.
Only then will it search hbase:meta for references to
'Unknown Servers'. Normal operation where we read Master
context is usual and sufficient. The scan of hbase:meta
is only for case where Master state has been corrupted
and we need to clear out 'Unknown Servers'.
2020-01-02 14:33:24 -08:00
Viraj Jasani 4d0ccec4ec
HBASE-23588 : Cache index & bloom blocks on write if CacheCompactedBlocksOnWrite is enabled
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: chenxu14 <47170471+chenxu14@users.noreply.github.com>
2020-01-02 20:41:47 +05:30
Manu Manjunath 30ef70728e
HBASE-23629: Add to 'Supporting Projects' in site
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-02 13:52:47 +05:30
Duo Zhang bee7f4e08c HBASE-23624 Add a tool to dump the procedure info in HFile (#975)
Signed-off-by: stack <stack@apache.org>
2020-01-02 11:32:29 +08:00
Jan Hentschel b81685cfcc
HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-02 00:42:13 +01:00
Jan Hentschel a18a5c4baa
HBASE-23627 Resolved remaining Checkstyle violations in hbase-thrift
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-01-01 22:56:17 +01:00
Viraj Jasani 17652a7b32
HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRefCount for auto region recovery based on old reader references
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-01 22:50:37 +05:30
binlijin 5a0dd574a6 HBASE-23615 Use a dedicated thread for executing WorkerMonitor in Pro… (#961)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: virajjasani <34790606+virajjasani@users.noreply.github.com>
2019-12-31 10:04:11 +08:00
Jan Hentschel 71a0c30caf
HBASE-23626 Reduced number of Checkstyle violations in tests in hbase-common
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-29 22:42:41 +01:00
Duo Zhang 7056238d26 HBASE-23618 Addendum add main method 2019-12-29 22:12:51 +08:00
Duo Zhang 2830c96254 HBASE-23618 Add a tool to dump procedure info in the WAL file (#969)
Signed-off-by: stack <stack@apache.org>
2019-12-29 21:50:24 +08:00
Jan Hentschel f3e11d7691
HBASE-23622 Reduced the number of Checkstyle violations in hbase-common
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-28 20:05:50 +01:00
Jan Hentschel 2382d009a6
HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-28 18:51:33 +01:00
Duo Zhang cfe6ccc755 HBASE-23617 Add a stress test tool for region based procedure store (#962)
Signed-off-by: stack <stack@apache.org>
2019-12-27 22:56:03 +08:00
Jan Hentschel de157b975b
HBASE-23619 Used built-in formatting for logger in hbase-zookeeper
Signed-off-by: stack <stack@apache.org>
2019-12-27 10:42:32 +01:00
Bharath Vissapragada 0cb9107e2d HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFilter (addendum) (#963)
Having it as static means the test cannot be parameterized (ran into
this issue in HBASE-23305). That happens because the field is not
reset between parameterized runs.
2019-12-26 11:16:44 -08:00
chenxu14 da4f756e6a
HBASE-23374 ExclusiveMemHFileBlock’s allocator should not be hardcoded as ByteBuffAllocator.HEAP
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2019-12-25 21:39:14 +01:00
Duo Zhang 5cae75e124 HBASE-23326 Implement a ProcedureStore which stores procedures in a HRegion (#941)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-12-25 12:21:26 +08:00
binlijin 976563e094 HBASE-23613 ProcedureExecutor check StuckWorkers blocked by DeadServe… (#960)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-25 10:00:37 +08:00
BukrosSzabolcs 0c09c43203 HBASE-23589: FlushDescriptor contains non-matching family/output combinations (#949)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-12-24 19:37:57 +08:00
Toshihiro Suzuki 0936bc23c8 HBASE-23581 Creating table gets stuck when specifying an invalid split policy as METADATA (#942)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Xu Cang <xucang@apache.org>
2019-12-24 19:25:40 +08:00
ravowlga123 7a4623f4fa HBASE-23320 Upgrade surefire plugin to 3.0.0-M4
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-12-23 14:55:45 +01:00
Ankit Singhal 6e6c7b3c2d HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-12-22 20:13:50 -08:00
stack 2e7a7cadd5 HBASE-23572 In 'HBCK Report', distringush between live, dead, and unknown servers
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-20 10:53:08 -08:00
binlijin 7a0e4d8140 HBASE-23564 RegionStates may has some expired serverinfo and make regions do not balance. (#930)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-12-20 15:09:57 +08:00
Duo Zhang 7a8d502a6e HBASE-23594 Procedure stuck due to region happen to recorded on two servers. (#953)
Signed-off-by: stack <stack@apache.org>
2019-12-20 13:43:50 +08:00
XinSun c9ab545f54 HBASE-23376 NPE happens while replica region is moving (#906)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-19 15:11:57 +08:00
Duo Zhang 7a1e9ca397 HBASE-20461 Implement fsync for AsyncFSWAL (#947)
Signed-off-by: stack <stack@apache.org>
2019-12-19 14:04:12 +08:00
Jacob Leblanc af0ce53836 HBASE-23066 Create a config that forces to cache blocks on compaction 2019-12-17 20:44:12 +05:30
Sean Busbey bb72dddb5e HBASE-23549 Document steps to disable MOB for a column family (#928)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit 17e180e4ee)
2019-12-16 13:04:17 -06:00
Sean Busbey 79134c8a3e HBASE-23239 Reporting on status of backing MOB files from client-facing cells (#785)
* Adds a new MapReduce job that builds a report of health of mob files
* Also builds background information on mob system use
* add a basic mob architecture in the ref guide to explain how mob takes the reference hfile value and finds the actual cell contents
* add a troubleshooting mob section to the ref guide to explain how to do a mob reference scan.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit bc2f162749)
2019-12-16 13:04:10 -06:00
Bharath Vissapragada a3fcc8badb HBASE-23575 Remove dead code in AsyncRegistry (#940)
Removes a bunch of dead code and fixes some checkstyle nits.

(cherry picked from commit efa4fe901a)
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-12-16 10:25:09 -06:00
Junegunn Choi dcd02d1236
HBASE-23582 Unbalanced braces in string representation of table descriptor
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2019-12-16 10:37:23 +01:00
David Mollitor acb6b7c1a9 HBASE-23380 General cleanup of FSUtil (#912)
* Clean up JavaDocs
* Clean up logging and error messages
* Remove superfluous code
* Replace static code with library call
* Do not swallow Interrupted Exceptions
* Use try-with-resources
* User multi-Exception catches to reduce code size

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 3be8ae2167)
2019-12-13 16:37:42 -06:00
BukrosSzabolcs f9abaee50c HBASE-23566: Fix package/packet terminology problem in chaos monkeys (#933)
s/package/packet/g

Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 413d4b2d0f)
2019-12-12 16:34:31 -06:00
meiyi e1301d46fd HBASE-23555 TestQuotaThrottle is broken (#924) 2019-12-11 12:23:52 -08:00
stack 362aba71ff HBASE-23554 Encoded regionname to regionname utility (#923); ADDENDUM 2019-12-11 09:51:50 -08:00
Karthik Palanisamy 719a043a8d HBASE-23360 [CLI] Fix help command 'set_quota' for removing limits (#896)
(cherry picked from commit a553b78c1c)
2019-12-11 11:07:07 -06:00
Michael Stack 18e3b48eb3 HBASE-23554 Encoded regionname to regionname utility (#923)
Adds shell command regioninfo:

  hbase(main):001:0>  regioninfo '0e6aa5c19ae2b2627649dc7708ce27d0'
  {ENCODED => 0e6aa5c19ae2b2627649dc7708ce27d0, NAME => 'TestTable,,1575941375972.0e6aa5c19ae2b2627649dc7708ce27d0.', STARTKEY => '', ENDKEY => '00000000000000000000299441'}
  Took 0.4737 seconds

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-11 08:57:24 -08:00
meiyi 0379d5c833 HBASE-23553 Snapshot referenced data files are deleted in some case (#922) 2019-12-11 09:52:19 +08:00
belugabehr d6eec74273 HBASE-23556: Minor ChoreService Cleanup (#927) 2019-12-10 16:21:11 -08:00
Nick Dimiduk 391be59835 HBASE-23552 Format Javadocs on ITBLL
We have this nice description in the java doc on ITBLL but it's
unformatted and thus illegible. Add some formatting so that it can be
read by humans.

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-12-10 13:12:54 -08:00
belugabehr f2230a822f
HBASE-23379 Clean Up FSUtil getRegionLocalityMappingFromFS
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-12-10 22:05:23 +01:00
binlijin 661580a032 HBASE-23377 Balancer should skip disabled tables's regions (#908)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-12-10 19:31:25 +08:00