Huaxiang Sun
920cf9faf4
HBASE-25126 Add load balance logic in hbase-client to distribute read load over meta replica regions
It adds load balance support for meta lookup in AsyncTableRegionLocator. The existing meta replica mode is renamed as "HedgedRead", client sends scan request to the primary meta replica region first, if response is not back within a configured amount of time, it will send scan requests to all meta replica regions and take the first response. On top of the existing mode, a new mode "LoadBalance" is introduced. In this mode, client first choose a meta replica region to send scan request. If the response is stale, it may send the request to another meta replica region or the primary region. In this mode, meta scan requests are load balanced across all replica regions with the primary mode as the ultimate source of truth. Two new config knobs are added: 1. hbase.locator.meta.replicas.mode Valid options are "None", "HedgedRead" and "LoadBalance", they are case insensitive. The default mode is "None". 2. hbase.locator.meta.replicas.mode.loadbalance.selector The load balance alogrithm to select a meta replica to send the requests. Only org.apache.hadoop.hbase.client.CatalogReplicaLoadBalanceReplicaSimpleSelector.class is supported for now, which is the default as well. The algorithm works as follows: a. Clients select a randome meta replica region to send the requests if there is no entry for the location in the stale location cache. b. If the location from one meta replica region is stale, a stale entry will be created in the statle location cache for the region. c. Clients select the primary meta region if the location is in the stale location cache. d. The stale location cache entries time out in 3 seconds. If there is no "hbase.locator.meta.replicas.mode" configured, it will check the config knob "hbase.meta.replicas.use". If "hbase.meta.replicas.use" is configured, the mode will be set to "HedgedRead". For branch-2 support, it introduces support for ReversedScan over a specific non-default replica region, this is mainly for load balance meta scan among all its replica regions. Remove ConnectionImplementation#setUseMetaReplicas()
HBASE-25126 Add load balance logic in hbase-client to distribute read load over meta replica regions
HBASE-25037 Lots of thread pool are changed to non daemon after HBASE-24750 which causes trouble when shutting down (#2407)
HBASE-25126 Add load balance logic in hbase-client to distribute read load over meta replica regions
Apache HBase [1] is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al.[2] Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Apache Hadoop [3]. To get started using HBase, the full documentation for this release can be found under the doc/ directory that accompanies this README. Using a browser, open the docs/index.html to view the project home page (or browse to [1]). The hbase 'book' at http://hbase.apache.org/book.html has a 'quick start' section and is where you should being your exploration of the hbase project. The latest HBase can be downloaded from an Apache Mirror [4]. The source code can be found at [5] The HBase issue tracker is at [6] Apache HBase is made available under the Apache License, version 2.0 [7] The HBase mailing lists and archives are listed here [8]. The HBase distribution includes cryptographic software. See the export control notice here [9]. 1. http://hbase.apache.org 2. http://research.google.com/archive/bigtable.html 3. http://hadoop.apache.org 4. http://www.apache.org/dyn/closer.cgi/hbase/ 5. https://hbase.apache.org/source-repository.html 6. https://hbase.apache.org/issue-tracking.html 7. http://hbase.apache.org/license.html 8. http://hbase.apache.org/mail-lists.html 9. https://hbase.apache.org/export_control.html
Description
Languages
Java
96.1%
Ruby
1.7%
Perl
0.8%
Shell
0.7%
Python
0.3%
Other
0.1%