7864 Commits

Author SHA1 Message Date
binlijin
1415a82d41 HBASE-23636 Disable table may hang when regionserver stop or abort. (#982)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: virajjasani <34790606+virajjasani@users.noreply.github.com>
2020-01-06 15:22:58 +08:00
Viraj Jasani
27660549d1
HBASE-23588 : Cache index & bloom blocks on write (ADDENDUM)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-06 12:07:05 +05:30
stack
c44a5c47dd HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionServers
Master force-closes unknown/incorrect Regions OPEN on RS

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Added a note and small refactor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Fix an NPE when CJ ran.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Minor clean up of log message; make it clearer.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 Make it so closeRegionSilentlyAndWait can be used w/o timeout.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 If a RegionServer Report notes a Region is OPEN and the Master does not
 know of said Region, close it (We used to crash out the RegionServer)

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateNode.java
 Minor tweak of toString -- label should be state, not rit (confusing).

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
 Doc.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
 Add region name to exception.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Be more careful about which Regions we queue up for reassign. This
 procedure is run by the operator so could happen at any time. We
 will likely be running this when Master has some accounting of
 cluster members so check its answers for what Regions were on
 server before running.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Doc and we were misrepresenting the case where a Region as not in RIT
 when we got CLOSE -- we were reporting it as though it was already
 trying to CLOSE.
2020-01-03 10:21:55 -08:00
Duo Zhang
26b1695df5 HBASE-23587 The FSYNC_WAL flag does not work on branch-2.x (#974)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-01-03 17:01:40 +08:00
Guanghao Zhang
a693a8fd95 HBASE-23286 Improve MTTR: Split WAL to HFile (#820)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-03 16:39:06 +08:00
Yiran Wu
381ff85e3d HBASE-23098 [bulkload] If one of the peers in a cluster is configured with NAMESPACE level, its hfile-refs(zk) will be backlogged (#676)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 Signed-off-by: stack <stack@apache.org>
2020-01-02 21:13:08 -08:00
Michael Stack
8cf5d045e0 HBASE-23632 DeadServer cleanup (#979)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-02 20:23:52 -08:00
stack
1b366719a0 HBASE-23596 HBCKServerCrashProcedure can double assign
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Change its behavior so it will only look in hbase:meta
if the call to the super class turns up zero references.
Only then will it search hbase:meta for references to
'Unknown Servers'. Normal operation where we read Master
context is usual and sufficient. The scan of hbase:meta
is only for case where Master state has been corrupted
and we need to clear out 'Unknown Servers'.
2020-01-02 14:33:24 -08:00
Viraj Jasani
4d0ccec4ec
HBASE-23588 : Cache index & bloom blocks on write if CacheCompactedBlocksOnWrite is enabled
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: chenxu14 <47170471+chenxu14@users.noreply.github.com>
2020-01-02 20:41:47 +05:30
Duo Zhang
bee7f4e08c HBASE-23624 Add a tool to dump the procedure info in HFile (#975)
Signed-off-by: stack <stack@apache.org>
2020-01-02 11:32:29 +08:00
Viraj Jasani
17652a7b32
HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRefCount for auto region recovery based on old reader references
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-01 22:50:37 +05:30
binlijin
5a0dd574a6 HBASE-23615 Use a dedicated thread for executing WorkerMonitor in Pro… (#961)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: virajjasani <34790606+virajjasani@users.noreply.github.com>
2019-12-31 10:04:11 +08:00
Duo Zhang
7056238d26 HBASE-23618 Addendum add main method 2019-12-29 22:12:51 +08:00
Duo Zhang
2830c96254 HBASE-23618 Add a tool to dump procedure info in the WAL file (#969)
Signed-off-by: stack <stack@apache.org>
2019-12-29 21:50:24 +08:00
Duo Zhang
cfe6ccc755 HBASE-23617 Add a stress test tool for region based procedure store (#962)
Signed-off-by: stack <stack@apache.org>
2019-12-27 22:56:03 +08:00
Bharath Vissapragada
0cb9107e2d HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFilter (addendum) (#963)
Having it as static means the test cannot be parameterized (ran into
this issue in HBASE-23305). That happens because the field is not
reset between parameterized runs.
2019-12-26 11:16:44 -08:00
chenxu14
da4f756e6a
HBASE-23374 ExclusiveMemHFileBlock’s allocator should not be hardcoded as ByteBuffAllocator.HEAP
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2019-12-25 21:39:14 +01:00
Duo Zhang
5cae75e124 HBASE-23326 Implement a ProcedureStore which stores procedures in a HRegion (#941)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-12-25 12:21:26 +08:00
binlijin
976563e094 HBASE-23613 ProcedureExecutor check StuckWorkers blocked by DeadServe… (#960)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-25 10:00:37 +08:00
BukrosSzabolcs
0c09c43203 HBASE-23589: FlushDescriptor contains non-matching family/output combinations (#949)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-12-24 19:37:57 +08:00
Toshihiro Suzuki
0936bc23c8 HBASE-23581 Creating table gets stuck when specifying an invalid split policy as METADATA (#942)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Xu Cang <xucang@apache.org>
2019-12-24 19:25:40 +08:00
Ankit Singhal
6e6c7b3c2d HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-12-22 20:13:50 -08:00
stack
2e7a7cadd5 HBASE-23572 In 'HBCK Report', distringush between live, dead, and unknown servers
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-20 10:53:08 -08:00
binlijin
7a0e4d8140 HBASE-23564 RegionStates may has some expired serverinfo and make regions do not balance. (#930)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-12-20 15:09:57 +08:00
Duo Zhang
7a8d502a6e HBASE-23594 Procedure stuck due to region happen to recorded on two servers. (#953)
Signed-off-by: stack <stack@apache.org>
2019-12-20 13:43:50 +08:00
Duo Zhang
7a1e9ca397 HBASE-20461 Implement fsync for AsyncFSWAL (#947)
Signed-off-by: stack <stack@apache.org>
2019-12-19 14:04:12 +08:00
Jacob Leblanc
af0ce53836 HBASE-23066 Create a config that forces to cache blocks on compaction 2019-12-17 20:44:12 +05:30
Sean Busbey
79134c8a3e HBASE-23239 Reporting on status of backing MOB files from client-facing cells (#785)
* Adds a new MapReduce job that builds a report of health of mob files
* Also builds background information on mob system use
* add a basic mob architecture in the ref guide to explain how mob takes the reference hfile value and finds the actual cell contents
* add a troubleshooting mob section to the ref guide to explain how to do a mob reference scan.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit bc2f16274930e272767ce20ede27eb8fa4117241)
2019-12-16 13:04:10 -06:00
Bharath Vissapragada
a3fcc8badb HBASE-23575 Remove dead code in AsyncRegistry (#940)
Removes a bunch of dead code and fixes some checkstyle nits.

(cherry picked from commit efa4fe901a3c050b0db087c45d47d250c7ca1039)
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-12-16 10:25:09 -06:00
David Mollitor
acb6b7c1a9 HBASE-23380 General cleanup of FSUtil (#912)
* Clean up JavaDocs
* Clean up logging and error messages
* Remove superfluous code
* Replace static code with library call
* Do not swallow Interrupted Exceptions
* Use try-with-resources
* User multi-Exception catches to reduce code size

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 3be8ae2167bc5433e6f71540e115727f71b040cf)
2019-12-13 16:37:42 -06:00
meiyi
e1301d46fd HBASE-23555 TestQuotaThrottle is broken (#924) 2019-12-11 12:23:52 -08:00
Michael Stack
18e3b48eb3 HBASE-23554 Encoded regionname to regionname utility (#923)
Adds shell command regioninfo:

  hbase(main):001:0>  regioninfo '0e6aa5c19ae2b2627649dc7708ce27d0'
  {ENCODED => 0e6aa5c19ae2b2627649dc7708ce27d0, NAME => 'TestTable,,1575941375972.0e6aa5c19ae2b2627649dc7708ce27d0.', STARTKEY => '', ENDKEY => '00000000000000000000299441'}
  Took 0.4737 seconds

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-11 08:57:24 -08:00
meiyi
0379d5c833 HBASE-23553 Snapshot referenced data files are deleted in some case (#922) 2019-12-11 09:52:19 +08:00
belugabehr
f2230a822f
HBASE-23379 Clean Up FSUtil getRegionLocalityMappingFromFS
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-12-10 22:05:23 +01:00
binlijin
661580a032 HBASE-23377 Balancer should skip disabled tables's regions (#908)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-12-10 19:31:25 +08:00
binlijin
751cc62611 HBASE-22280 Separate read/write handler for priority request(especial… (#202)
Signed-off-by: Yu Li <liyu@apache.org>
2019-12-09 16:19:57 +08:00
Lars Hofhansl
5afaff66dc HBASE-23364 HRegionServer sometimes does not shut down. 2019-12-05 18:24:57 -08:00
Karthik Palanisamy
393c7ddec4 HBASE-23361 Limit two decimals in total average load (#897)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by: Xu Cang <xucang@apache.org>
2019-12-05 10:19:47 +08:00
binlijin
b39fe1aefa HBASE-23365 Minor change MemStoreFlusher's log (#900)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by: Xu Cang <xucang@apache.org>
2019-12-05 10:13:52 +08:00
Junegunn Choi
40e1c9e310 HBASE-22529 Add sanity check for in-memory compaction policy
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-12-04 21:05:20 +09:00
Bharath Vissapragada
e87db35d1a HBASE-23362: [WalPrettyPrinter] print/filter by table name. (#898)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 570cc3c4a4d30bdd3dd11e6b4db4329fe207925c)
2019-12-04 10:15:06 +00:00
binlijin
2e8ea9e272 HBASE-23356 When construct StoreScanner throw exceptions it is possible to left some KeyValueScanner not closed. (#891)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
2019-12-04 10:36:44 +08:00
ddupg
997684f24d HBASE-23345 Table need to replication unless all of cfs are excluded
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-12-04 09:19:38 +08:00
Toshihiro Suzuki
e558fc9cd8 HBASE-22096 /storeFile.jsp shows CorruptHFileException when the storeFile is a reference file (#888)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-12-03 15:36:29 +09:00
Guanghao Zhang
71c913f4da HBASE-23298 Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink (#832)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-02 18:40:45 +08:00
Viraj Jasani
399fc8d098 HBASE-23335 : Improving cost functions array copy in StochasticLoadBalancer (#874) 2019-12-01 21:43:16 -08:00
chenxu14
46c090ce45 HBASE-23293 [REPLICATION] make ship edits timeout configurable (#882)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2019-11-28 11:01:18 +08:00
Wellington Ramos Chevreuil
70bbc38aae HBASE-23313 [hbck2] setRegionState should update Master in-memory sta… (#864)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-11-27 12:15:11 +00:00
Sandeep Pal
ab09e74055 HBASE-23117: Bad enum in hbase:meta info:state column can fail loadMeta and stop startup (#867)
* Handling the BAD value in info:state columns in hbase:meta

* Adding a unit test and region encoded name in the LOG

* Adding a null check for region state to complete the test scenario and fixing the nit

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by: stack <stack@apache.org>
2019-11-26 10:10:01 -08:00
Wellington Ramos Chevreuil
5635752179 HBASE-23197 'IllegalArgumentException: Wrong FS' on edits replay when… (#740)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-11-25 08:33:58 +00:00