HBASE-13699 Expand documentation about quotas and other load balancing mechanisms
This commit is contained in:
Misty Stanley-Jones
parent
88f0f421c3
commit
a93353e83c
|
|
@ -1609,282 +1609,186 @@ You can use the HBase Shell command `status 'replication'` to monitor the replic
|
|||
* `status 'replication', 'source'` -- prints the status for each replication source, sorted by hostname.
|
||||
* `status 'replication', 'sink'` -- prints the status for each replication sink, sorted by hostname.
|
||||
|
||||
== Running Multiple Workloads On a Single Cluster
|
||||
|
||||
HBase provides the following mechanisms for managing the performance of a cluster
|
||||
handling multiple workloads:
|
||||
. <<quota>>
|
||||
. <<request-queues>>
|
||||
. <<multiple-typed-queues>>
|
||||
|
||||
[[quota]]
|
||||
== HBase Quota
|
||||
=== Quotas
|
||||
HBASE-11598 introduces quotas, which allow you to throttle requests based on
|
||||
the following limits:
|
||||
|
||||
When a work load increases on a cluster due to multiple users or user requests, the system needs to prioritize the users or the user requests for a smooth operation.
|
||||
This is handled by the HBase quota feature. The quota allows the admin to limit the number of tables, regions, or request in a system.
|
||||
. <<request-quotas,The number or size of requests in a given timeframe>>
|
||||
. <<namespace-quotas,The number of tables allowed in a namespace>>
|
||||
|
||||
The HBase features three types of quotas, they are:
|
||||
These limits can be enforced for a specified user, table, or namespace.
|
||||
|
||||
. Namespace Quota
|
||||
. Request Number Quota
|
||||
. Request Size Quota
|
||||
.Enabling Quotas
|
||||
|
||||
[[enabling.quota]]
|
||||
=== Enabling Quota
|
||||
Quotas are disabled by default. To enable the feature, set the `hbase.quota.enabled`
|
||||
property to `true` in _hbase-site.xml_ file for all cluster nodes.
|
||||
|
||||
By default the quota function is disabled. To enable the quota, change the value of parameter `hbase.quota.enabled` to `true` in _hbase-site.xml_ file for all HMaster and HRegionServer machines.
|
||||
.General Quota Syntax
|
||||
. Timeframes can be expressed in the following units: `sec`, `min`, `hour`, `day`
|
||||
. Request sizes can be expressed in the following units: `B` (bytes), `K` (kilobytes),
|
||||
`M` (megabytes), `G` (gigabytes), `T` (terabytes), `P` (petabytes)
|
||||
. Numbers of requests are expressed as an integer followed by the string `req`
|
||||
. Limits relating to time are expressed as req/time or size/time. For instance `10req/day`
|
||||
or `100P/hour`.
|
||||
. Numbers of tables or regions are expressed as integers.
|
||||
|
||||
[[quota.cache.refresh]]
|
||||
=== Quota Cache Refresh Configuration
|
||||
By default quota setting cache refresh time period `'hbase.quota.refresh.period'` is set to 5*60000(5 mins). Which means a user has to wait for maximum of 5 mins to get his quota settings updated.
|
||||
[[request-quotas]]
|
||||
.Setting Request Quotas
|
||||
You can set quota rules ahead of time, or you can change the throttle at runtime. The change
|
||||
will propagate after the quota refresh period has expired. This expiration period
|
||||
defaults to 5 minutes. To change it, modify the `hbase.quota.refresh.period` property
|
||||
in `hbase-site.xml`. This property is expressed in milliseconds and defaults to `300000`.
|
||||
|
||||
----
|
||||
# Limit user u1 to 10 requests per second
|
||||
hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => '10req/sec'
|
||||
|
||||
# Limit user u1 to 10 M per day everywhere
|
||||
hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => '10M/day'
|
||||
|
||||
# Limit user u1 to 5k per minute on table t2
|
||||
hbase> set_quota TYPE => THROTTLE, USER => 'u1', TABLE => 't2', LIMIT => '5K/min'
|
||||
|
||||
# Remove an existing limit from user u1 on namespace ns2
|
||||
hbase> set_quota TYPE => THROTTLE, USER => 'u1', NAMESPACE => 'ns2', LIMIT => NONE
|
||||
|
||||
# Limit all users to 10 requests per hour on namespace ns1
|
||||
hbase> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => '10req/shour'
|
||||
|
||||
# Limit all users to 10 T per hour on table t1
|
||||
hbase> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT => '10T/hour'
|
||||
|
||||
# Remove all existing limits from user u1
|
||||
hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => NONE
|
||||
|
||||
# List all quotas for user u1 in namespace ns2
|
||||
hbase> list_quotas USER => 'u1, NAMESPACE => 'ns2'
|
||||
|
||||
# List all quotas for namespace ns2
|
||||
hbase> list_quotas NAMESPACE => 'ns2'
|
||||
|
||||
# List all quotas for table t1
|
||||
hbase> list_quotas TABLE => 't1'
|
||||
|
||||
# list all quotas
|
||||
hbase> list_quotas
|
||||
----
|
||||
|
||||
You can also place a global limit and exclude a user or a table from the limit by applying the
|
||||
`GLOBAL_BYPASS` property.
|
||||
----
|
||||
hbase> set_quota NAMESPACE => 'ns1', LIMIT => '100req/min' # a per-namespace request limit
|
||||
hbase> set_quota USER => 'u1', GLOBAL_BYPASS => true # user u1 is not affected by the limit
|
||||
----
|
||||
|
||||
[[namespace_quotas]]
|
||||
.Setting Namespace Quotas
|
||||
|
||||
You can specify the maximum number of tables or regions allowed in a given namespace, either
|
||||
when you create the namespace or by altering an existing namespace, by setting the
|
||||
`hbase.namespace.quota.maxtables property` on the namespace.
|
||||
|
||||
.Limiting Tables Per Namespace
|
||||
----
|
||||
# Create a namespace with a max of 5 tables
|
||||
hbase> create_namespace 'ns1', {'hbase.namespace.quota.maxtables'=>'5'}
|
||||
|
||||
# Alter an existing namespace to have a max of 8 tables
|
||||
hbase> alter_namespace 'ns2', {METHOD => 'set', 'hbase.namespace.quota.maxtables'=>'8'}
|
||||
|
||||
# Show quota information for a namespace
|
||||
hbase> describe_namespace 'ns2'
|
||||
|
||||
# Alter an existing namespace to remove a quota
|
||||
hbase> alter_namespace 'ns2', {METHOD => 'unset', NAME=>'hbase.namespace.quota.maxtables'}
|
||||
----
|
||||
|
||||
.Limiting Regions Per Namespace
|
||||
----
|
||||
# Create a namespace with a max of 10 regions
|
||||
hbase> create_namespace 'ns1', {'hbase.namespace.quota.maxregions'=>'10'
|
||||
|
||||
# Show quota information for a namespace
|
||||
hbase> describe_namespace 'ns1'
|
||||
|
||||
# Alter an existing namespace to have a max of 20 tables
|
||||
hbase> alter_namespace 'ns2', {METHOD => 'set', 'hbase.namespace.quota.maxregions'=>'20'}
|
||||
|
||||
# Alter an existing namespace to remove a quota
|
||||
hbase> alter_namespace 'ns2', {METHOD => 'unset', NAME=> 'hbase.namespace.quota.maxregions'}
|
||||
----
|
||||
|
||||
[[request_queues]]
|
||||
=== Request Queues
|
||||
If no throttling policy is configured, when the RegionServer receives multiple requests,
|
||||
they are now placed into a queue waiting for a free execution slot (HBASE-6721).
|
||||
The simplest queue is a FIFO queue, where each request waits for all previous requests in the queue
|
||||
to finish before running. Fast or interactive queries can get stuck behind large requests.
|
||||
|
||||
If you are able to guess how long a request will take, you can reorder requests by
|
||||
pushing the long requests to the end of the queue and allowing short requests to preempt
|
||||
them. Eventually, you must still execute the large requests and prioritize the new
|
||||
requests behind them. The short requests will be newer, so the result is not terrible,
|
||||
but still suboptimal compared to a mechanism which allows large requests to be split
|
||||
into multiple smaller ones.
|
||||
|
||||
HBASE-10993 introduces such a system for deprioritizing long-running scanners. There
|
||||
are two types of queues,`fifo` and `deadline`.To configure the type of queue used,
|
||||
configure the `hbase.ipc.server.callqueue.type` property in `hbase-site.xml`. There
|
||||
is no way to estimate how long each request may take, so de-prioritization only affects
|
||||
scans, and is based on the number of “next” calls a scan request has made. An assumption
|
||||
is made that when you are doing a full table scan, your job is not likely to be interactive,
|
||||
so if there are concurrent requests, you can delay long-running scans up to a limit tunable by
|
||||
setting the `hbase.ipc.server.queue.max.call.delay` property. The slope of the delay is calculated
|
||||
by a simple square root of `(numNextCall * weight)` where the weight is
|
||||
configurable by setting the `hbase.ipc.server.scan.vtime.weight` property.
|
||||
|
||||
[[multiple-typed-queues]]
|
||||
=== Multiple-Typed Queues
|
||||
|
||||
You can also prioritize or deprioritize different kinds of requests by configuring
|
||||
a specified number of dedicated handlers and queues. You can segregate the scan requests
|
||||
in a single queue with a single handler, and all the other available queues can service
|
||||
short `Get` requests.
|
||||
|
||||
You can adjust the IPC queues and handlers based on the type of workload, using static
|
||||
tuning options. This approach is an interim first step that will eventually allow
|
||||
you to change the settings at runtime, and to dynamically adjust values based on the load.
|
||||
|
||||
.Multiple Queues
|
||||
|
||||
To avoid contention and separate different kinds of requests, configure the
|
||||
`hbase.ipc.server.callqueue.handler.factor` property, which allows you to increase the number of
|
||||
queues and control how many handlers can share the same queue., allows admins to increase the number
|
||||
of queues and decide how many handlers share the same queue.
|
||||
|
||||
Using more queues reduces contention when adding a task to a queue or selecting it
|
||||
from a queue. You can even configure one queue per handler. The trade-off is that
|
||||
if some queues contain long-running tasks, a handler may need to wait to execute from that queue
|
||||
rather than stealing from another queue which has waiting tasks.
|
||||
|
||||
.Read and Write Queues
|
||||
With multiple queues, you can now divide read and write requests, giving more priority
|
||||
(more queues) to one or the other type. Use the `hbase.ipc.server.callqueue.read.ratio`
|
||||
property to choose to serve more reads or more writes.
|
||||
|
||||
.Get and Scan Queues
|
||||
Similar to the read/write split, you can split gets and scans by tuning the `hbase.ipc.server.callqueue.scan.ratio`
|
||||
property to give more priority to gets or to scans. A scan ratio of `0.1` will give
|
||||
more queue/handlers to the incoming gets, which means that more gets can be processed
|
||||
at the same time and that fewer scans can be executed at the same time. A value of
|
||||
`0.9` will give more queue/handlers to scans, so the number of scans executed will
|
||||
increase and the number of gets will decrease.
|
||||
|
||||
[[namespace.quota]]
|
||||
=== Namespace Quota
|
||||
|
||||
Namespace quota reserves the number of tables or regions allowed in a Namespace.
|
||||
|
||||
Following are the namespace quota:
|
||||
|
||||
* Number of tables allowed in a namespace.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| create_namespace 'namespace_name', {'hbase.namespace.quota.maxtables'=>'5'}
|
||||
| Creates a namespace with the maximum table quota.
|
||||
|
||||
| describe_namespace 'namespace_name'
|
||||
| Displays the quota information on the namespace.
|
||||
|
||||
| alter_namespace 'namespace_name ', {METHOD => 'set', 'hbase.namespace.quota.maxtables'=>'8'}
|
||||
| Modifies the existing namespace and sets the maximum table quota.
|
||||
|
||||
| alter_namespace 'namespace_name ', {METHOD => 'unset', NAME=> 'hbase.namespace.quota.maxtables'}
|
||||
| Removes the maximum table quota set on a namespace.
|
||||
|
||||
|===
|
||||
|
||||
* Number of regions allowed in a namespace.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| create_namespace 'namespace_name', {'hbase.namespace.quota.maxregions'=>'10'}
|
||||
| Creates a namespace with the maximum regions quota.
|
||||
|
||||
| describe_namespace 'namespace_name '
|
||||
| Displays the quota information on the namespace.
|
||||
|
||||
| alter_namespace 'namespace_name ', {METHOD => 'set', 'hbase.namespace.quota.maxregions'=>'20'}
|
||||
| Modifies the existing namespace and sets the maximum regions quota.
|
||||
|
||||
| alter_namespace 'namespace_name ', {METHOD => 'unset', NAME=> 'hbase.namespace.quota.maxregions'}
|
||||
| Removes the maximum regions quota set on a namespace.
|
||||
|
||||
|===
|
||||
|
||||
[[number.quota]]
|
||||
=== Request Number Quota
|
||||
|
||||
Request Number Quota reserves the number of requests that is allowed to execute on a particular table or namespace in any given time by a user or users with similar name.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The Request Number Quota uses time units in the command.
|
||||
The valid time units are sec, min, hour, and day.
|
||||
====
|
||||
|
||||
Following are the five types of request number quota:
|
||||
|
||||
* Number of requests a user can execute in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'user_name', LIMIT => '10req/sec'
|
||||
| Sets 10req/sec quota for a specified user.
|
||||
|
||||
| list_quotas USER => 'user_name'
|
||||
| Displays the quota information of the specified user.
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => ' user_name ', LIMIT => NONE
|
||||
| Removes the quota of the specified user.
|
||||
|
||||
|===
|
||||
|
||||
* Number of requests a user can execute on a given table in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => ' user_name ', TABLE => 't1', LIMIT => '10req/sec'
|
||||
| Sets 10req/sec quota on a particular table for a specified user.
|
||||
|
||||
| list_quotas USER => 'user_name', TABLE => 't1'
|
||||
| Displays the quota information of the specified user on a particular table.
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'u1', TABLE => 't1', LIMIT => NONE
|
||||
| Removes the quota on a particular table for a specified user.
|
||||
|
||||
|===
|
||||
|
||||
* Number of requests a user can execute on a given namespace in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => ' user_name ', NAMESPACE => 'ns1', LIMIT => '10req/sec'
|
||||
| Sets 10req/sec quota on a particular namespace for a specified user.
|
||||
|
||||
| list_quotas USER => 'user_name', NAMESPACE => 'ns.*'
|
||||
| Displays the quota information of the specified user on a particular namespace.
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'u1', NAMESPACE => 'ns1', LIMIT => NONE
|
||||
| Removes the quota on a particular namespace for a specified user.
|
||||
|
||||
|===
|
||||
|
||||
* Number of requests that can be allowed on a table in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, TABLE => 't1, LIMIT => '10req/sec'
|
||||
| Sets 10req/sec quota on a particular table.
|
||||
|
||||
| list_quotas TABLE => 't1'
|
||||
| Displays the quota information on a particular table.
|
||||
|
||||
| set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT => NONE
|
||||
| Removes the quota on a particular table.
|
||||
|
||||
|===
|
||||
|
||||
* Number of requests that can be allowed on a namespace in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => '10req/sec'
|
||||
| Sets 10req/sec quota on a particular namespace.
|
||||
|
||||
| list_quotas NAMESPACE => 'ns1'
|
||||
| Displays the quota information on a particular namespace.
|
||||
|
||||
| set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => NONE
|
||||
| Removes the quota on a particular namespace.
|
||||
|
||||
|===
|
||||
|
||||
[[size.quota]]
|
||||
=== Request Size Quota
|
||||
|
||||
Request size quota reserves the size of the requests that is allowed to execute on a particular table or a namespace in any given time by a user or users with similar name.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The Request Size Quota uses time and size unit in the command.
|
||||
The valid size units are B(byte), K(kilobyte), M(megabyte), G(gigabyte), T(terabyte), and P(petabyte).
|
||||
====
|
||||
|
||||
Following are the five types of request size quota:
|
||||
|
||||
* Size of requests a user can execute in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'user_name', LIMIT => '5K/min'
|
||||
| Sets 5k/sec quota for a specified user.
|
||||
|
||||
| list_quotas USER => 'user_name'
|
||||
| Displays the quota information of the specified user.
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => ' user_name ', LIMIT => NONE
|
||||
| Removes the quota of the specified user.
|
||||
|
||||
|===
|
||||
|
||||
* Size of requests a user can execute on a given table in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'u1', TABLE => 'table_name', LIMIT => '5K/min'
|
||||
| Sets 5k/sec quota on a particular table for a specified user.
|
||||
|
||||
| list_quotas USER => 'bob.*', TABLE => 't1'
|
||||
| Displays the quota information of the specified user on a particular table.
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'u1', TABLE => 't1', LIMIT => NONE
|
||||
| Removes the quota on a particular table for a specified user.
|
||||
|
||||
|===
|
||||
|
||||
* Size of requests a user can execute on a given namespace in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'u1', NAMESPACE => 'ns1', LIMIT => '5K/min'
|
||||
| Sets 5k/sec quota on a particular namespace for a specified user.
|
||||
|
||||
| list_quotas USER => 'bob.*', NAMESPACE => 'ns.*'
|
||||
| Displays the quota information of the specified user on a particular namespace.
|
||||
|
||||
| set_quota TYPE => THROTTLE, USER => 'u1', NAMESPACE => 'ns1', LIMIT => NONE
|
||||
| Removes the quota on a particular namespace for a specified user.
|
||||
|
||||
|===
|
||||
|
||||
* Size of requests that can be allowed on a table in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT => '5K/min'
|
||||
| Sets 5k/sec quota on a particular table.
|
||||
|
||||
| list_quotas TABLE => 'myTable'
|
||||
| Displays the quota information on a particular table.
|
||||
|
||||
| set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT => NONE
|
||||
| Removes the quota on a particular table.
|
||||
|
||||
|===
|
||||
|
||||
* Size of requests that can be allowed on a namespace in a given time.
|
||||
|
||||
[cols="1,1", options="header"]
|
||||
|===
|
||||
| Example
|
||||
| Command Description
|
||||
|
||||
| set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => '5K/min'
|
||||
| Sets 5k/sec quota on a particular namespace.
|
||||
|
||||
| list_quotas NAMESPACE => 'ns.*'
|
||||
| Displays the quota information on a particular namespace.
|
||||
|
||||
| set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => NONE
|
||||
| Removes the quota on a particular namespace.
|
||||
|
||||
|===
|
||||
|
||||
[[ops.backup]]
|
||||
== HBase Backup
|
||||
|
|
|
|||
Loading…
Reference in New Issue