HBASE-14053 Disable DLR in branch-1+
This commit is contained in:
stack
parent
22f0f3026f
commit
8da1214099
|
|
@ -875,11 +875,9 @@ public final class HConstants {
|
|||
|
||||
public static final String LOCALHOST_IP = "127.0.0.1";
|
||||
|
||||
/** Conf key that enables unflushed WAL edits directly being replayed to region servers */
|
||||
public static final String DISTRIBUTED_LOG_REPLAY_KEY = "hbase.master.distributed.log.replay";
|
||||
/**
|
||||
* Default 'distributed log replay' as true since hbase 1.1 (HBASE-12577)
|
||||
/** Conf key that enables unflushed WAL edits directly being replayed to region servers
|
||||
*/
|
||||
public static final String DISTRIBUTED_LOG_REPLAY_KEY = "hbase.master.distributed.log.replay";
|
||||
public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
|
||||
public static final String DISALLOW_WRITES_IN_RECOVERING =
|
||||
"hbase.regionserver.disallow.writes.when.recovering";
|
||||
|
|
|
|||
|
|
@ -283,17 +283,6 @@ possible configurations would overwhelm and obscure the important.
|
|||
<value>org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter</value>
|
||||
<description>The WAL file writer implementation.</description>
|
||||
</property>
|
||||
<property>
|
||||
<name>hbase.master.distributed.log.replay</name>
|
||||
<value>true</value>
|
||||
<description>Enable 'distributed log replay' as default engine splitting
|
||||
WAL files on server crash. This default is new in hbase 1.0. To fall
|
||||
back to the old mode 'distributed log splitter', set the value to
|
||||
'false'. 'Disributed log replay' improves MTTR because it does not
|
||||
write intermediate files. 'DLR' required that 'hfile.format.version'
|
||||
be set to version 3 or higher.
|
||||
</description>
|
||||
</property>
|
||||
<property>
|
||||
<name>hbase.regionserver.global.memstore.size</name>
|
||||
<value></value>
|
||||
|
|
@ -309,7 +298,7 @@ possible configurations would overwhelm and obscure the important.
|
|||
<value></value>
|
||||
<description>Maximum size of all memstores in a region server before flushes are forced.
|
||||
Defaults to 95% of hbase.regionserver.global.memstore.size (0.95).
|
||||
A 100% value for this value causes the minimum possible flushing to occur when updates are
|
||||
A 100% value for this value causes the minimum possible flushing to occur when updates are
|
||||
blocked due to memstore limiting.
|
||||
The default value in this configuration has been intentionally left emtpy in order to
|
||||
honor the old hbase.regionserver.global.memstore.lowerLimit property if present.</description>
|
||||
|
|
@ -754,9 +743,9 @@ possible configurations would overwhelm and obscure the important.
|
|||
<property>
|
||||
<name>hbase.hstore.time.to.purge.deletes</name>
|
||||
<value>0</value>
|
||||
<description>The amount of time to delay purging of delete markers with future timestamps. If
|
||||
unset, or set to 0, all delete markers, including those with future timestamps, are purged
|
||||
during the next major compaction. Otherwise, a delete marker is kept until the major compaction
|
||||
<description>The amount of time to delay purging of delete markers with future timestamps. If
|
||||
unset, or set to 0, all delete markers, including those with future timestamps, are purged
|
||||
during the next major compaction. Otherwise, a delete marker is kept until the major compaction
|
||||
which occurs after the marker's timestamp plus the value of this setting, in milliseconds.
|
||||
</description>
|
||||
</property>
|
||||
|
|
@ -800,7 +789,7 @@ possible configurations would overwhelm and obscure the important.
|
|||
<description>The HFile format version to use for new files.
|
||||
Version 3 adds support for tags in hfiles (See http://hbase.apache.org/book.html#hbase.tags).
|
||||
Distributed Log Replay requires that tags are enabled. Also see the configuration
|
||||
'hbase.replication.rpc.codec'.
|
||||
'hbase.replication.rpc.codec'.
|
||||
</description>
|
||||
</property>
|
||||
<property>
|
||||
|
|
@ -1210,7 +1199,7 @@ possible configurations would overwhelm and obscure the important.
|
|||
fails, we will switch back to using HDFS checksums (so do not disable HDFS
|
||||
checksums! And besides this feature applies to hfiles only, not to WALs).
|
||||
If this parameter is set to false, then hbase will not verify any checksums,
|
||||
instead it will depend on checksum verification being done in the HDFS client.
|
||||
instead it will depend on checksum verification being done in the HDFS client.
|
||||
</description>
|
||||
</property>
|
||||
<property>
|
||||
|
|
@ -1344,11 +1333,11 @@ possible configurations would overwhelm and obscure the important.
|
|||
<property>
|
||||
<name>hbase.procedure.regionserver.classes</name>
|
||||
<value></value>
|
||||
<description>A comma-separated list of
|
||||
org.apache.hadoop.hbase.procedure.RegionServerProcedureManager procedure managers that are
|
||||
loaded by default on the active HRegionServer process. The lifecycle methods (init/start/stop)
|
||||
will be called by the active HRegionServer process to perform the specific globally barriered
|
||||
procedure. After implementing your own RegionServerProcedureManager, just put it in
|
||||
<description>A comma-separated list of
|
||||
org.apache.hadoop.hbase.procedure.RegionServerProcedureManager procedure managers that are
|
||||
loaded by default on the active HRegionServer process. The lifecycle methods (init/start/stop)
|
||||
will be called by the active HRegionServer process to perform the specific globally barriered
|
||||
procedure. After implementing your own RegionServerProcedureManager, just put it in
|
||||
HBase's classpath and add the fully qualified class name here.
|
||||
</description>
|
||||
</property>
|
||||
|
|
@ -1389,8 +1378,8 @@ possible configurations would overwhelm and obscure the important.
|
|||
which will tail the logs and replicate the mutatations to region replicas for tables that
|
||||
have region replication > 1. If this is enabled once, disabling this replication also
|
||||
requires disabling the replication peer using shell or ReplicationAdmin java class.
|
||||
Replication to secondary region replicas works over standard inter-cluster replication.
|
||||
So replication, if disabled explicitly, also has to be enabled by setting "hbase.replication"
|
||||
Replication to secondary region replicas works over standard inter-cluster replication.
|
||||
So replication, if disabled explicitly, also has to be enabled by setting "hbase.replication"
|
||||
to true for this feature to work.
|
||||
</description>
|
||||
</property>
|
||||
|
|
@ -1398,12 +1387,12 @@ possible configurations would overwhelm and obscure the important.
|
|||
<name>hbase.http.filter.initializers</name>
|
||||
<value>org.apache.hadoop.hbase.http.lib.StaticUserWebFilter</value>
|
||||
<description>
|
||||
A comma separated list of class names. Each class in the list must extend
|
||||
org.apache.hadoop.hbase.http.FilterInitializer. The corresponding Filter will
|
||||
be initialized. Then, the Filter will be applied to all user facing jsp
|
||||
and servlet web pages.
|
||||
A comma separated list of class names. Each class in the list must extend
|
||||
org.apache.hadoop.hbase.http.FilterInitializer. The corresponding Filter will
|
||||
be initialized. Then, the Filter will be applied to all user facing jsp
|
||||
and servlet web pages.
|
||||
The ordering of the list defines the ordering of the filters.
|
||||
The default StaticUserWebFilter add a user principal as defined by the
|
||||
The default StaticUserWebFilter add a user principal as defined by the
|
||||
hbase.http.staticuser.user property.
|
||||
</description>
|
||||
</property>
|
||||
|
|
@ -1419,7 +1408,7 @@ possible configurations would overwhelm and obscure the important.
|
|||
<name>hbase.http.max.threads</name>
|
||||
<value>10</value>
|
||||
<description>
|
||||
The maximum number of threads that the HTTP Server will create in its
|
||||
The maximum number of threads that the HTTP Server will create in its
|
||||
ThreadPool.
|
||||
</description>
|
||||
</property>
|
||||
|
|
@ -1428,7 +1417,7 @@ possible configurations would overwhelm and obscure the important.
|
|||
<value>org.apache.hadoop.hbase.codec.KeyValueCodecWithTags</value>
|
||||
<description>
|
||||
The codec that is to be used when replication is enabled so that
|
||||
the tags are also replicated. This is used along with HFileV3 which
|
||||
the tags are also replicated. This is used along with HFileV3 which
|
||||
supports tags in them. If tags are not used or if the hfile version used
|
||||
is HFileV2 then KeyValueCodec can be used as the replication codec. Note that
|
||||
using KeyValueCodecWithTags for replication when there are no tags causes no harm.
|
||||
|
|
|
|||
|
|
@ -2690,13 +2690,6 @@ rs.close();
|
|||
|
||||
<section>
|
||||
<title>Performance Improvements during Log Splitting</title>
|
||||
<para>
|
||||
WAL log splitting and recovery can be resource intensive and take a long time,
|
||||
depending on the number of RegionServers involved in the crash and the size of the
|
||||
regions. <xref linkend="distributed.log.splitting" /> and <xref
|
||||
linkend="distributed.log.replay" /> were developed to improve
|
||||
performance during log splitting.
|
||||
</para>
|
||||
<section xml:id="distributed.log.splitting">
|
||||
<title>Distributed Log Splitting</title>
|
||||
<para><firstterm>Distributed Log Splitting</firstterm> was added in HBase version 0.92
|
||||
|
|
@ -2907,37 +2900,6 @@ ctime = Sat Jun 23 11:13:40 PDT 2012
|
|||
</step>
|
||||
</procedure>
|
||||
</section>
|
||||
<section xml:id="distributed.log.replay">
|
||||
<title>Distributed Log Replay</title>
|
||||
<para>After a RegionServer fails, its failed region is assigned to another
|
||||
RegionServer, which is marked as "recovering" in ZooKeeper. A split log worker directly
|
||||
replays edits from the WAL of the failed region server to the region at its new
|
||||
location. When a region is in "recovering" state, it can accept writes but no reads
|
||||
(including Append and Increment), region splits or merges. </para>
|
||||
<para>Distributed Log Replay extends the <xref linkend="distributed.log.splitting" /> framework. It works by
|
||||
directly replaying WAL edits to another RegionServer instead of creating
|
||||
<filename>recovered.edits</filename> files. It provides the following advantages
|
||||
over distributed log splitting alone:</para>
|
||||
<itemizedlist>
|
||||
<listitem><para>It eliminates the overhead of writing and reading a large number of
|
||||
<filename>recovered.edits</filename> files. It is not unusual for thousands of
|
||||
<filename>recovered.edits</filename> files to be created and written concurrently
|
||||
during a RegionServer recovery. Many small random writes can degrade overall
|
||||
system performance.</para></listitem>
|
||||
<listitem><para>It allows writes even when a region is in recovering state. It only takes seconds for a recovering region to accept writes again.
|
||||
</para></listitem>
|
||||
</itemizedlist>
|
||||
<formalpara>
|
||||
<title>Enabling Distributed Log Replay</title>
|
||||
<para>To enable distributed log replay, set <varname>hbase.master.distributed.log.replay</varname> to
|
||||
true. This will be the default for HBase 0.99 (<link
|
||||
xlink:href="https://issues.apache.org/jira/browse/HBASE-10888">HBASE-10888</link>).</para>
|
||||
</formalpara>
|
||||
<para>You must also enable HFile version 3 (which is the default HFile format starting
|
||||
in HBase 0.99. See <link
|
||||
xlink:href="https://issues.apache.org/jira/browse/HBASE-10855">HBASE-10855</link>).
|
||||
Distributed log replay is unsafe for rolling upgrades.</para>
|
||||
</section>
|
||||
</section>
|
||||
</section>
|
||||
<section xml:id="wal.disable">
|
||||
|
|
|
|||
Loading…
Reference in New Issue