7643 Commits

Author SHA1 Message Date
Ankit Singhal
6e6c7b3c2d HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-12-22 20:13:50 -08:00
stack
2e7a7cadd5 HBASE-23572 In 'HBCK Report', distringush between live, dead, and unknown servers
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-20 10:53:08 -08:00
binlijin
7a0e4d8140 HBASE-23564 RegionStates may has some expired serverinfo and make regions do not balance. (#930)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-12-20 15:09:57 +08:00
Duo Zhang
7a8d502a6e HBASE-23594 Procedure stuck due to region happen to recorded on two servers. (#953)
Signed-off-by: stack <stack@apache.org>
2019-12-20 13:43:50 +08:00
Duo Zhang
7a1e9ca397 HBASE-20461 Implement fsync for AsyncFSWAL (#947)
Signed-off-by: stack <stack@apache.org>
2019-12-19 14:04:12 +08:00
Jacob Leblanc
af0ce53836 HBASE-23066 Create a config that forces to cache blocks on compaction 2019-12-17 20:44:12 +05:30
Sean Busbey
79134c8a3e HBASE-23239 Reporting on status of backing MOB files from client-facing cells (#785)
* Adds a new MapReduce job that builds a report of health of mob files
* Also builds background information on mob system use
* add a basic mob architecture in the ref guide to explain how mob takes the reference hfile value and finds the actual cell contents
* add a troubleshooting mob section to the ref guide to explain how to do a mob reference scan.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit bc2f16274930e272767ce20ede27eb8fa4117241)
2019-12-16 13:04:10 -06:00
Bharath Vissapragada
a3fcc8badb HBASE-23575 Remove dead code in AsyncRegistry (#940)
Removes a bunch of dead code and fixes some checkstyle nits.

(cherry picked from commit efa4fe901a3c050b0db087c45d47d250c7ca1039)
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-12-16 10:25:09 -06:00
David Mollitor
acb6b7c1a9 HBASE-23380 General cleanup of FSUtil (#912)
* Clean up JavaDocs
* Clean up logging and error messages
* Remove superfluous code
* Replace static code with library call
* Do not swallow Interrupted Exceptions
* Use try-with-resources
* User multi-Exception catches to reduce code size

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 3be8ae2167bc5433e6f71540e115727f71b040cf)
2019-12-13 16:37:42 -06:00
meiyi
e1301d46fd HBASE-23555 TestQuotaThrottle is broken (#924) 2019-12-11 12:23:52 -08:00
Michael Stack
18e3b48eb3 HBASE-23554 Encoded regionname to regionname utility (#923)
Adds shell command regioninfo:

  hbase(main):001:0>  regioninfo '0e6aa5c19ae2b2627649dc7708ce27d0'
  {ENCODED => 0e6aa5c19ae2b2627649dc7708ce27d0, NAME => 'TestTable,,1575941375972.0e6aa5c19ae2b2627649dc7708ce27d0.', STARTKEY => '', ENDKEY => '00000000000000000000299441'}
  Took 0.4737 seconds

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-11 08:57:24 -08:00
meiyi
0379d5c833 HBASE-23553 Snapshot referenced data files are deleted in some case (#922) 2019-12-11 09:52:19 +08:00
belugabehr
f2230a822f
HBASE-23379 Clean Up FSUtil getRegionLocalityMappingFromFS
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-12-10 22:05:23 +01:00
binlijin
661580a032 HBASE-23377 Balancer should skip disabled tables's regions (#908)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-12-10 19:31:25 +08:00
binlijin
751cc62611 HBASE-22280 Separate read/write handler for priority request(especial… (#202)
Signed-off-by: Yu Li <liyu@apache.org>
2019-12-09 16:19:57 +08:00
Lars Hofhansl
5afaff66dc HBASE-23364 HRegionServer sometimes does not shut down. 2019-12-05 18:24:57 -08:00
Karthik Palanisamy
393c7ddec4 HBASE-23361 Limit two decimals in total average load (#897)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by: Xu Cang <xucang@apache.org>
2019-12-05 10:19:47 +08:00
binlijin
b39fe1aefa HBASE-23365 Minor change MemStoreFlusher's log (#900)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by: Xu Cang <xucang@apache.org>
2019-12-05 10:13:52 +08:00
Junegunn Choi
40e1c9e310 HBASE-22529 Add sanity check for in-memory compaction policy
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-12-04 21:05:20 +09:00
Bharath Vissapragada
e87db35d1a HBASE-23362: [WalPrettyPrinter] print/filter by table name. (#898)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 570cc3c4a4d30bdd3dd11e6b4db4329fe207925c)
2019-12-04 10:15:06 +00:00
binlijin
2e8ea9e272 HBASE-23356 When construct StoreScanner throw exceptions it is possible to left some KeyValueScanner not closed. (#891)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
2019-12-04 10:36:44 +08:00
ddupg
997684f24d HBASE-23345 Table need to replication unless all of cfs are excluded
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-12-04 09:19:38 +08:00
Toshihiro Suzuki
e558fc9cd8 HBASE-22096 /storeFile.jsp shows CorruptHFileException when the storeFile is a reference file (#888)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-12-03 15:36:29 +09:00
Guanghao Zhang
71c913f4da HBASE-23298 Refactor LogRecoveredEditsOutputSink and BoundedLogWriterCreationOutputSink (#832)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-02 18:40:45 +08:00
Viraj Jasani
399fc8d098 HBASE-23335 : Improving cost functions array copy in StochasticLoadBalancer (#874) 2019-12-01 21:43:16 -08:00
chenxu14
46c090ce45 HBASE-23293 [REPLICATION] make ship edits timeout configurable (#882)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2019-11-28 11:01:18 +08:00
Wellington Ramos Chevreuil
70bbc38aae HBASE-23313 [hbck2] setRegionState should update Master in-memory sta… (#864)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-11-27 12:15:11 +00:00
Sandeep Pal
ab09e74055 HBASE-23117: Bad enum in hbase:meta info:state column can fail loadMeta and stop startup (#867)
* Handling the BAD value in info:state columns in hbase:meta

* Adding a unit test and region encoded name in the LOG

* Adding a null check for region state to complete the test scenario and fixing the nit

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by: stack <stack@apache.org>
2019-11-26 10:10:01 -08:00
Wellington Ramos Chevreuil
5635752179 HBASE-23197 'IllegalArgumentException: Wrong FS' on edits replay when… (#740)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-11-25 08:33:58 +00:00
Karthik Palanisamy
1fa9a8c62a HBASE-23237 Prevent Negative values in metrics requestsPerSecond (#865)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-11-23 23:27:49 +08:00
Peter Somogyi
c60d11b2ba HBASE-22969 A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position; ADDENDUM (#869)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-11-22 13:51:49 -06:00
Michael Stack
c8592f1fb7 HBASE-23307 Add running of ReplicationBarrierCleaner to hbck2 fixMeta invocation (#859)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-11-22 08:27:58 -08:00
xuqinya1
f905bb32c3 HBASE-23325 [UI]rsgoup average load keep two decimals (#860)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2019-11-22 14:34:09 +08:00
Nick Dimiduk
d742112ed9
HBASE-23259: Populate master address end points in cluster/rs configs (#807) (#857)
All the clients need to know the master RPC end points while using master
based registry for creating cluster connections. This patch amends the
test cluster utility to populate these configs in the base configuration
object used to spin up the cluster.

The config key added here ("hbase.master.addrs") is used in the subsequent
patches for HBASE-18095.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2019-11-21 09:43:53 -08:00
stack
126ea28c80 HBASE-23322 [hbck2] Simplification on HBCKSCP scheduling
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-11-20 21:25:26 -08:00
stack
7e088daf39 HBASE-23321 [hbck2] fixHoles of fixMeta doesn't update in-memory state 2019-11-20 09:02:41 -08:00
stack
a05cef75c4 Add status when fixing hole
TODO: Add region to master memory and region state.
2019-11-20 09:02:41 -08:00
stack
70771b603e HBASE-23315 Miscellaneous HBCK Report page cleanup
* Add a bit of javadoc around SerialReplicationChecker.
 * Miniscule edit to the profiler jsp page and then a bit of doc on how to make it work that might help.
 * Add some detail if NPE getting BitSetNode to help w/ debug.
 * Change HbckChore to log region names instead of encoded names; helps doing diagnostics; can take region name and query in shell to find out all about the region according to hbase:meta.
 * Add some fix-it help inline in the HBCK Report page – how to fix.
 * Add counts in procedures page so can see if making progress; move listing of WALs to end of the page.
2019-11-19 07:33:13 -08:00
Baiqiang Zhao
c6ad71e256 HBASE-23278 Add a table-level compaction progress display on the UI (#816)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2019-11-19 19:38:43 +08:00
stack
44c8b58cec HBASE-23282 HBCKServerCrashProcedure for 'Unknown Servers'
Have the existing scheduleRecoveries launch a new HBCKSCP
instead of SCP. It gets regions to recover from Master
in-memory context AND from a scan of hbase:meta. This
new HBCKSCP is For processing 'Unknown Servers', servers that
are 'dead' and purged but still have references in
hbase:meta. Rare occurance but needs tooling to address.
Later have catalogjanitor take care of these deviations
between Master in-memory and hbase:meta content (usually
because of overdriven cluster with failed RPCs to hbase:meta,
etc)

Changed expireServers in ServerManager so could pass in
custom reaction to expired server.... This is how we
run our custom HBCKSCP while keeping all other aspects
of expiring services (rather than try replicate it
externally).
2019-11-18 14:26:11 -08:00
Udai Bhan Kashyap
ab63bde013 HBASE-22969 A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position (#829)
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2019-11-18 13:30:08 +01:00
Duo Zhang
e69431ae8c HBASE-23294 ReplicationBarrierCleaner should delete all the barriers for a removed region which does not belong to any serial replication peer (#827)
Signed-off-by: stack <stack@apache.org>
2019-11-15 19:04:42 +08:00
Geoffrey Jacoby
da06aa7f1f HBASE-23251 - Add Column Family and Table Names to HFileContext and use in HFileWriterImpl logging (#796)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-11-11 16:22:09 +08:00
Nick Dimiduk
f42b2aa108
HBASE-23230 Enforce member visibility in HRegionServer (#775) (#806)
* Clean up a bunch of private variable leakage into other
   classes. Reduces visibility as much as possible, providing getters
   where access remains necessary or making use of getters that
   already exist. There remains an insidious relationship between
   `HRegionServer` and `RSRpcServices`.
 * Rename `fs` to `dataFs`, `rootDir` as `dataRootDir` so as to
   distinguish from the new `walFs`, `walRootDir` (and make it easier
   to spot bugs).
 * Cleanup or delete a bunch of lack-luster javadoc comments.
 * Delete a handful of methods that are unused according to static
   analysis.
 * Reduces the warning count as reported by IntelliJ from 100 to 7.

Signed-off-by: stack <stack@apache.org>
2019-11-08 13:54:02 -08:00
chenxu14
235d03bc07 HBASE-22888 Share some stuffs with the initial reader when new stream reader created (#581)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-11-08 14:30:49 +08:00
Karthik Palanisamy
10b38d5f82 HBASE-23263 NPE in Quotas.jsp (#800)
Signed-off-by: Guangxu Cheng <guangxucheng@gmail.com>
2019-11-08 09:56:35 +08:00
Karthik Palanisamy
0a20c4010f HBASE-23262 Cannot load Master UI (#799)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-11-08 09:50:32 +08:00
lijinbin
e8c16a2fb9 HBASE-22480 Get block from BlockCache once and return this block to BlockCache twice make ref count error. 2019-11-08 09:35:37 +08:00
Viraj Jasani
29c27e3d04
HBASE-23212 Dynamically reload configs for Region Recovery chore (#802)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-11-06 15:54:57 -08:00
ravowlga123
d4c23ee0c6
HBASE-23250 Log message about CleanerChore delegate initialization should be at INFO
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-11-05 20:23:37 +01:00