Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-11339 Addendum: Put back the sweeper tool docs for now

This commit is contained in:
Misty Stanley-Jones 2015-03-09 09:16:59 +10:00
parent 8b4671c10e
commit 33a6a819a4
1 changed files with 64 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

64
src/main/asciidoc/_chapters/hbase_mob.adoc
View File

@ -118,3 +118,67 @@ Because there can be a large number of MOB files at any time, as compared to the
---- ----
==== ====
=== MOB Optimization Tasks
HBase MOB currently relies on a MapReduce job called the Sweeper tool for optimization. The Sweeper tool oalesces small MOB files or MOB files with many deletions or updates. A native MOB compaction tool is still in testing. To configure the Sweeper tool, set the following options:
[source,xml]
----
<property>
<name>hbase.mob.sweep.tool.compaction.ratio</name>
<value>0.5f</value>
<description>
If there are too many cells deleted in a mob file, it's regarded
as an invalid file and needs to be merged.
If existingCellsSize/mobFileSize is less than ratio, it's regarded
as an invalid file. The default value is 0.5f.
</description>
</property>
<property>
<name>hbase.mob.sweep.tool.compaction.mergeable.size</name>
<value>134217728</value>
<description>
If the size of a mob file is less than this value, it's regarded as a small
file and needs to be merged. The default value is 128MB.
</description>
</property>
<property>
<name>hbase.mob.sweep.tool.compaction.memstore.flush.size</name>
<value>134217728</value>
<description>
The flush size for the memstore used by sweep job. Each sweep reducer owns such a memstore.
The default value is 128MB.
</description>
</property>
<property>
<name>hbase.mob.cleaner.interval</name>
<value>86400000</value>
<description>
The period that ExpiredMobFileCleaner runs. The unit is millisecond.
The default value is one day.
</description>
</property>
----
Next, add the HBase install directory, _`$HBASE_HOME`/*_, and HBase library directory to _yarn-site.xml_ Adjust this example to suit your environment.
[source,xml]
----
<property>
<description>Classpath for typical applications.</description>
<name>yarn.application.classpath</name>
<value>
$HADOOP_CONF_DIR,
$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*,
$HBASE_HOME/*, $HBASE_HOME/lib/*
</value>
</property>
----
Finally, run the `sweeper` tool for each column which is configured for MOB.
[source,bash]
----
$ org.apache.hadoop.hbase.mob.compactions.Sweeper _tableName_ _familyName_
----