Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-27803 Update ref guide about the changes of the replication storage related concepts (#5199) 

Signed-off-by: Liangjun He <heliangjun@apache.org>
This commit is contained in:
Duo Zhang 2023-04-25 21:51:51 +08:00 committed by GitHub
parent a1f2b4e1d0
commit 3593288d60
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 55 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
src/main/asciidoc/_chapters/ops_mgt.adoc
View File

@ -2292,6 +2292,33 @@ enable_table_replication <TABLE_NAME>::
Enable the table replication switch for all its column families. If the table is not found in the destination cluster then it will create one with the same name and column families.
disable_table_replication <TABLE_NAME>::
Disable the table replication switch for all its column families.
peer_modification_switch <enable_or_disable>, <drain_procedures>::
Enabled/Disable peer modification operations, such as adding/removing replication peers.
The second parameter means whether you want to wait until all existing peer modification
procedures to finish before returning when disabling peer modification.
peer_modification_enabled::
Check whether peer modification is enabled.
=== Migrate Across Different Replication Peer Storages
Starting from 2.6.0, we introduce a file system based `ReplicationPeerStorage`, which stores
the replication peer state with files on HFile file system, instead of znodes on ZooKeeper.
And we have also implemented a tool to copy replication peer state across different replication
peer storages.
[source,shell]
----
./bin/hbase copyreppeers <SRC_REPLICATION_PEER_STORAGE> <DST_REPLICATION_PEER_STORAGE>
----
To support doing the migrate online, we introduce a shell command called `peer_modification_switch`.
[source,ruby]
----
hbase> peer_modification_switch false, true
----
Use the above command can disable peer modification operations. The second `true` means you want to
wait until all the existing replication peer modification procedures to finish before returning.
After disabling the peer modification, it is safe for you to copy replication peer state with the
above tool, and then update all the _hbase-site.xml_ files in the cluster to specify the new
replication peer storage, and finally trigger an online configuration update to load the new
replication peer storage.
=== Serial Replication
@ -2401,10 +2428,18 @@ This option was introduced in link:https://issues.apache.org/jira/browse/HBASE-1
==== Replication Internals
Replication State Storage::
In HBASE-15867, we abstract two interfaces for storing replication state,
`ReplicationPeerStorage` and `ReplicationQueueStorage`. The former one is for storing the
replication peer related states, and the latter one is for storing the replication queue related
states.
HBASE-15867 is only half done, as although we have abstract these two interfaces, we still only
have zookeeper based implementations.
Replication State in ZooKeeper::
HBase replication maintains its state in ZooKeeper.
By default, the state is contained in the base node _/hbase/replication_.
This node contains two child nodes, the `Peers` znode and the `RS` znode.
Usually this nodes contains two child nodes, the `peers` znode is for storing replication peer
state, and the `rs` znodes is for storing replication queue state.
The `Peers` Znode::
The `peers` znode is stored in _/hbase/replication/peers_ by default.
@ -2419,6 +2454,15 @@ The `RS` Znode::
The child znode name is the region server's hostname, client port, and start code.
This list includes both live and dead region servers.
Other implementations for `ReplicationPeerStorage`::
Starting from 2.6.0, we introduce a file system based `ReplicationPeerStorage`, which stores
the replication peer state with files on HFile file system, instead of znodes on ZooKeeper.
The layout is almost the same with znodes on zookeeper, the main difference is that, the HFile
file system may not support atomic rename, so we use two files to store the state and when reading
we will read them both and compare the timestamp to find out the newer one. So typically, you will
see two peer config files. And for enable/disable state, we just touch a disabled file if the peer
is disabled, and remove the file when enabling the peer.
==== Choosing Region Servers to Replicate To
When a master cluster region server initiates a replication source to a slave cluster, it first connects to the slave's ZooKeeper ensemble using the provided cluster key . It then scans the _rs/_ directory to discover all the available sinks (region servers that are accepting incoming streams of edits to replicate) and randomly chooses a subset of them using a configured ratio which has a default value of 10%. For example, if a slave cluster has 150 machines, 15 will be chosen as potential recipient for edits that this master cluster region server sends.
@ -2641,6 +2685,15 @@ The following metrics are exposed at the global region server level and at the p
| The number of region servers a given region server should attempt to
failover simultaneously.
| 1
| hbase.replication.peer.storage.impl
| The replication peer storage implementation
| zookeeper
| hbase.replication.peers.directory
| The directory for storing replication peer state, when filesystem replication
peer storage is specified
| peers
|===
=== Monitoring Replication Status