Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-14071 Document troubleshooting unexpected filesystem usage by snapshots and WALs

This commit is contained in:
Misty Stanley-Jones 2015-08-10 09:04:48 +10:00
parent 0daae43342
commit b29c731a57
1 changed files with 25 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
src/main/asciidoc/_chapters/troubleshooting.adoc
View File

@ -842,6 +842,31 @@ Two common use-cases for querying HDFS for HBase objects is research the degree
If there are a large number of StoreFiles for each ColumnFamily it could indicate the need for a major compaction. If there are a large number of StoreFiles for each ColumnFamily it could indicate the need for a major compaction.
Additionally, after a major compaction if the resulting StoreFile is "small" it could indicate the need for a reduction of ColumnFamilies for the table. Additionally, after a major compaction if the resulting StoreFile is "small" it could indicate the need for a reduction of ColumnFamilies for the table.
=== Unexpected Filesystem Growth
If you see an unexpected spike in filesystem usage by HBase, two possible culprits
are snapshots and WALs.
Snapshots::
When you create a snapshot, HBase retains everything it needs to recreate the table's
state at that time of tne snapshot. This includes deleted cells or expired versions.
For this reason, your snapshot usage pattern should be well-planned, and you should
prune snapshots that you no longer need. Snapshots are stored in `/hbase/.snapshots`,
and archives needed to restore snapshots are stored in
`/hbase/.archive/<_tablename_>/<_region_>/<_column_family_>/`.
*Do not* manage snapshots or archives manually via HDFS. HBase provides APIs and
HBase Shell commands for managing them. For more information, see <<ops.snapshots>>.
WAL::
Write-ahead logs (WALs) are stored in subdirectories of `/hbase/.logs/`, depending
on their status. Already-processed WALs are stored in `/hbase/.logs/oldWALs/` and
corrupt WALs are stored in `/hbase/.logs/.corrupt/` for examination.
If the size of any subdirectory of `/hbase/.logs/` is growing, examine the HBase
server logs to find the root cause for why WALs are not being processed correctly.
*Do not* manage WALs manually via HDFS.
[[trouble.network]] [[trouble.network]]
== Network == Network