HBASE-14053 Disable DLR in branch-1+
This commit is contained in:
parent
22f0f3026f
commit
8da1214099
|
@ -875,11 +875,9 @@ public final class HConstants {
|
||||||
|
|
||||||
public static final String LOCALHOST_IP = "127.0.0.1";
|
public static final String LOCALHOST_IP = "127.0.0.1";
|
||||||
|
|
||||||
/** Conf key that enables unflushed WAL edits directly being replayed to region servers */
|
/** Conf key that enables unflushed WAL edits directly being replayed to region servers
|
||||||
public static final String DISTRIBUTED_LOG_REPLAY_KEY = "hbase.master.distributed.log.replay";
|
|
||||||
/**
|
|
||||||
* Default 'distributed log replay' as true since hbase 1.1 (HBASE-12577)
|
|
||||||
*/
|
*/
|
||||||
|
public static final String DISTRIBUTED_LOG_REPLAY_KEY = "hbase.master.distributed.log.replay";
|
||||||
public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
|
public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
|
||||||
public static final String DISALLOW_WRITES_IN_RECOVERING =
|
public static final String DISALLOW_WRITES_IN_RECOVERING =
|
||||||
"hbase.regionserver.disallow.writes.when.recovering";
|
"hbase.regionserver.disallow.writes.when.recovering";
|
||||||
|
|
|
@ -283,17 +283,6 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<value>org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter</value>
|
<value>org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter</value>
|
||||||
<description>The WAL file writer implementation.</description>
|
<description>The WAL file writer implementation.</description>
|
||||||
</property>
|
</property>
|
||||||
<property>
|
|
||||||
<name>hbase.master.distributed.log.replay</name>
|
|
||||||
<value>true</value>
|
|
||||||
<description>Enable 'distributed log replay' as default engine splitting
|
|
||||||
WAL files on server crash. This default is new in hbase 1.0. To fall
|
|
||||||
back to the old mode 'distributed log splitter', set the value to
|
|
||||||
'false'. 'Disributed log replay' improves MTTR because it does not
|
|
||||||
write intermediate files. 'DLR' required that 'hfile.format.version'
|
|
||||||
be set to version 3 or higher.
|
|
||||||
</description>
|
|
||||||
</property>
|
|
||||||
<property>
|
<property>
|
||||||
<name>hbase.regionserver.global.memstore.size</name>
|
<name>hbase.regionserver.global.memstore.size</name>
|
||||||
<value></value>
|
<value></value>
|
||||||
|
@ -309,7 +298,7 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<value></value>
|
<value></value>
|
||||||
<description>Maximum size of all memstores in a region server before flushes are forced.
|
<description>Maximum size of all memstores in a region server before flushes are forced.
|
||||||
Defaults to 95% of hbase.regionserver.global.memstore.size (0.95).
|
Defaults to 95% of hbase.regionserver.global.memstore.size (0.95).
|
||||||
A 100% value for this value causes the minimum possible flushing to occur when updates are
|
A 100% value for this value causes the minimum possible flushing to occur when updates are
|
||||||
blocked due to memstore limiting.
|
blocked due to memstore limiting.
|
||||||
The default value in this configuration has been intentionally left emtpy in order to
|
The default value in this configuration has been intentionally left emtpy in order to
|
||||||
honor the old hbase.regionserver.global.memstore.lowerLimit property if present.</description>
|
honor the old hbase.regionserver.global.memstore.lowerLimit property if present.</description>
|
||||||
|
@ -754,9 +743,9 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<property>
|
<property>
|
||||||
<name>hbase.hstore.time.to.purge.deletes</name>
|
<name>hbase.hstore.time.to.purge.deletes</name>
|
||||||
<value>0</value>
|
<value>0</value>
|
||||||
<description>The amount of time to delay purging of delete markers with future timestamps. If
|
<description>The amount of time to delay purging of delete markers with future timestamps. If
|
||||||
unset, or set to 0, all delete markers, including those with future timestamps, are purged
|
unset, or set to 0, all delete markers, including those with future timestamps, are purged
|
||||||
during the next major compaction. Otherwise, a delete marker is kept until the major compaction
|
during the next major compaction. Otherwise, a delete marker is kept until the major compaction
|
||||||
which occurs after the marker's timestamp plus the value of this setting, in milliseconds.
|
which occurs after the marker's timestamp plus the value of this setting, in milliseconds.
|
||||||
</description>
|
</description>
|
||||||
</property>
|
</property>
|
||||||
|
@ -800,7 +789,7 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<description>The HFile format version to use for new files.
|
<description>The HFile format version to use for new files.
|
||||||
Version 3 adds support for tags in hfiles (See http://hbase.apache.org/book.html#hbase.tags).
|
Version 3 adds support for tags in hfiles (See http://hbase.apache.org/book.html#hbase.tags).
|
||||||
Distributed Log Replay requires that tags are enabled. Also see the configuration
|
Distributed Log Replay requires that tags are enabled. Also see the configuration
|
||||||
'hbase.replication.rpc.codec'.
|
'hbase.replication.rpc.codec'.
|
||||||
</description>
|
</description>
|
||||||
</property>
|
</property>
|
||||||
<property>
|
<property>
|
||||||
|
@ -1210,7 +1199,7 @@ possible configurations would overwhelm and obscure the important.
|
||||||
fails, we will switch back to using HDFS checksums (so do not disable HDFS
|
fails, we will switch back to using HDFS checksums (so do not disable HDFS
|
||||||
checksums! And besides this feature applies to hfiles only, not to WALs).
|
checksums! And besides this feature applies to hfiles only, not to WALs).
|
||||||
If this parameter is set to false, then hbase will not verify any checksums,
|
If this parameter is set to false, then hbase will not verify any checksums,
|
||||||
instead it will depend on checksum verification being done in the HDFS client.
|
instead it will depend on checksum verification being done in the HDFS client.
|
||||||
</description>
|
</description>
|
||||||
</property>
|
</property>
|
||||||
<property>
|
<property>
|
||||||
|
@ -1344,11 +1333,11 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<property>
|
<property>
|
||||||
<name>hbase.procedure.regionserver.classes</name>
|
<name>hbase.procedure.regionserver.classes</name>
|
||||||
<value></value>
|
<value></value>
|
||||||
<description>A comma-separated list of
|
<description>A comma-separated list of
|
||||||
org.apache.hadoop.hbase.procedure.RegionServerProcedureManager procedure managers that are
|
org.apache.hadoop.hbase.procedure.RegionServerProcedureManager procedure managers that are
|
||||||
loaded by default on the active HRegionServer process. The lifecycle methods (init/start/stop)
|
loaded by default on the active HRegionServer process. The lifecycle methods (init/start/stop)
|
||||||
will be called by the active HRegionServer process to perform the specific globally barriered
|
will be called by the active HRegionServer process to perform the specific globally barriered
|
||||||
procedure. After implementing your own RegionServerProcedureManager, just put it in
|
procedure. After implementing your own RegionServerProcedureManager, just put it in
|
||||||
HBase's classpath and add the fully qualified class name here.
|
HBase's classpath and add the fully qualified class name here.
|
||||||
</description>
|
</description>
|
||||||
</property>
|
</property>
|
||||||
|
@ -1389,8 +1378,8 @@ possible configurations would overwhelm and obscure the important.
|
||||||
which will tail the logs and replicate the mutatations to region replicas for tables that
|
which will tail the logs and replicate the mutatations to region replicas for tables that
|
||||||
have region replication > 1. If this is enabled once, disabling this replication also
|
have region replication > 1. If this is enabled once, disabling this replication also
|
||||||
requires disabling the replication peer using shell or ReplicationAdmin java class.
|
requires disabling the replication peer using shell or ReplicationAdmin java class.
|
||||||
Replication to secondary region replicas works over standard inter-cluster replication.
|
Replication to secondary region replicas works over standard inter-cluster replication.
|
||||||
So replication, if disabled explicitly, also has to be enabled by setting "hbase.replication"
|
So replication, if disabled explicitly, also has to be enabled by setting "hbase.replication"
|
||||||
to true for this feature to work.
|
to true for this feature to work.
|
||||||
</description>
|
</description>
|
||||||
</property>
|
</property>
|
||||||
|
@ -1398,12 +1387,12 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<name>hbase.http.filter.initializers</name>
|
<name>hbase.http.filter.initializers</name>
|
||||||
<value>org.apache.hadoop.hbase.http.lib.StaticUserWebFilter</value>
|
<value>org.apache.hadoop.hbase.http.lib.StaticUserWebFilter</value>
|
||||||
<description>
|
<description>
|
||||||
A comma separated list of class names. Each class in the list must extend
|
A comma separated list of class names. Each class in the list must extend
|
||||||
org.apache.hadoop.hbase.http.FilterInitializer. The corresponding Filter will
|
org.apache.hadoop.hbase.http.FilterInitializer. The corresponding Filter will
|
||||||
be initialized. Then, the Filter will be applied to all user facing jsp
|
be initialized. Then, the Filter will be applied to all user facing jsp
|
||||||
and servlet web pages.
|
and servlet web pages.
|
||||||
The ordering of the list defines the ordering of the filters.
|
The ordering of the list defines the ordering of the filters.
|
||||||
The default StaticUserWebFilter add a user principal as defined by the
|
The default StaticUserWebFilter add a user principal as defined by the
|
||||||
hbase.http.staticuser.user property.
|
hbase.http.staticuser.user property.
|
||||||
</description>
|
</description>
|
||||||
</property>
|
</property>
|
||||||
|
@ -1419,7 +1408,7 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<name>hbase.http.max.threads</name>
|
<name>hbase.http.max.threads</name>
|
||||||
<value>10</value>
|
<value>10</value>
|
||||||
<description>
|
<description>
|
||||||
The maximum number of threads that the HTTP Server will create in its
|
The maximum number of threads that the HTTP Server will create in its
|
||||||
ThreadPool.
|
ThreadPool.
|
||||||
</description>
|
</description>
|
||||||
</property>
|
</property>
|
||||||
|
@ -1428,7 +1417,7 @@ possible configurations would overwhelm and obscure the important.
|
||||||
<value>org.apache.hadoop.hbase.codec.KeyValueCodecWithTags</value>
|
<value>org.apache.hadoop.hbase.codec.KeyValueCodecWithTags</value>
|
||||||
<description>
|
<description>
|
||||||
The codec that is to be used when replication is enabled so that
|
The codec that is to be used when replication is enabled so that
|
||||||
the tags are also replicated. This is used along with HFileV3 which
|
the tags are also replicated. This is used along with HFileV3 which
|
||||||
supports tags in them. If tags are not used or if the hfile version used
|
supports tags in them. If tags are not used or if the hfile version used
|
||||||
is HFileV2 then KeyValueCodec can be used as the replication codec. Note that
|
is HFileV2 then KeyValueCodec can be used as the replication codec. Note that
|
||||||
using KeyValueCodecWithTags for replication when there are no tags causes no harm.
|
using KeyValueCodecWithTags for replication when there are no tags causes no harm.
|
||||||
|
|
|
@ -2690,13 +2690,6 @@ rs.close();
|
||||||
|
|
||||||
<section>
|
<section>
|
||||||
<title>Performance Improvements during Log Splitting</title>
|
<title>Performance Improvements during Log Splitting</title>
|
||||||
<para>
|
|
||||||
WAL log splitting and recovery can be resource intensive and take a long time,
|
|
||||||
depending on the number of RegionServers involved in the crash and the size of the
|
|
||||||
regions. <xref linkend="distributed.log.splitting" /> and <xref
|
|
||||||
linkend="distributed.log.replay" /> were developed to improve
|
|
||||||
performance during log splitting.
|
|
||||||
</para>
|
|
||||||
<section xml:id="distributed.log.splitting">
|
<section xml:id="distributed.log.splitting">
|
||||||
<title>Distributed Log Splitting</title>
|
<title>Distributed Log Splitting</title>
|
||||||
<para><firstterm>Distributed Log Splitting</firstterm> was added in HBase version 0.92
|
<para><firstterm>Distributed Log Splitting</firstterm> was added in HBase version 0.92
|
||||||
|
@ -2907,37 +2900,6 @@ ctime = Sat Jun 23 11:13:40 PDT 2012
|
||||||
</step>
|
</step>
|
||||||
</procedure>
|
</procedure>
|
||||||
</section>
|
</section>
|
||||||
<section xml:id="distributed.log.replay">
|
|
||||||
<title>Distributed Log Replay</title>
|
|
||||||
<para>After a RegionServer fails, its failed region is assigned to another
|
|
||||||
RegionServer, which is marked as "recovering" in ZooKeeper. A split log worker directly
|
|
||||||
replays edits from the WAL of the failed region server to the region at its new
|
|
||||||
location. When a region is in "recovering" state, it can accept writes but no reads
|
|
||||||
(including Append and Increment), region splits or merges. </para>
|
|
||||||
<para>Distributed Log Replay extends the <xref linkend="distributed.log.splitting" /> framework. It works by
|
|
||||||
directly replaying WAL edits to another RegionServer instead of creating
|
|
||||||
<filename>recovered.edits</filename> files. It provides the following advantages
|
|
||||||
over distributed log splitting alone:</para>
|
|
||||||
<itemizedlist>
|
|
||||||
<listitem><para>It eliminates the overhead of writing and reading a large number of
|
|
||||||
<filename>recovered.edits</filename> files. It is not unusual for thousands of
|
|
||||||
<filename>recovered.edits</filename> files to be created and written concurrently
|
|
||||||
during a RegionServer recovery. Many small random writes can degrade overall
|
|
||||||
system performance.</para></listitem>
|
|
||||||
<listitem><para>It allows writes even when a region is in recovering state. It only takes seconds for a recovering region to accept writes again.
|
|
||||||
</para></listitem>
|
|
||||||
</itemizedlist>
|
|
||||||
<formalpara>
|
|
||||||
<title>Enabling Distributed Log Replay</title>
|
|
||||||
<para>To enable distributed log replay, set <varname>hbase.master.distributed.log.replay</varname> to
|
|
||||||
true. This will be the default for HBase 0.99 (<link
|
|
||||||
xlink:href="https://issues.apache.org/jira/browse/HBASE-10888">HBASE-10888</link>).</para>
|
|
||||||
</formalpara>
|
|
||||||
<para>You must also enable HFile version 3 (which is the default HFile format starting
|
|
||||||
in HBase 0.99. See <link
|
|
||||||
xlink:href="https://issues.apache.org/jira/browse/HBASE-10855">HBASE-10855</link>).
|
|
||||||
Distributed log replay is unsafe for rolling upgrades.</para>
|
|
||||||
</section>
|
|
||||||
</section>
|
</section>
|
||||||
</section>
|
</section>
|
||||||
<section xml:id="wal.disable">
|
<section xml:id="wal.disable">
|
||||||
|
|
Loading…
Reference in New Issue