From f9b99d2f24db5faae3ded11c3b74240943b1e49e Mon Sep 17 00:00:00 2001 From: Siyao Meng Date: Thu, 31 Oct 2019 12:59:06 -0700 Subject: [PATCH] HADOOP-16656. Document FairCallQueue configs in core-default.xml. Contributed by Siyao Meng. Signed-off-by: Wei-Chiu Chuang --- .../src/main/resources/core-default.xml | 187 ++++++++++++++++++ 1 file changed, 187 insertions(+) diff --git a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml index 6e5f400c23f..f0aa44dc999 100644 --- a/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml +++ b/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml @@ -2274,6 +2274,193 @@ because the server side is stuck in TIME_WAIT state. + + + + + + + + + + + + + ipc.[port_number].backoff.enable + false + Whether or not to enable client backoff when a queue is full. + + + + + ipc.[port_number].callqueue.impl + java.util.concurrent.LinkedBlockingQueue + The fully qualified name of a class to use as the implementation + of a call queue. The default implementation is + java.util.concurrent.LinkedBlockingQueue (FIFO queue). + Use org.apache.hadoop.ipc.FairCallQueue for the Fair Call Queue. + + + + + ipc.[port_number].scheduler.impl + org.apache.hadoop.ipc.DefaultRpcScheduler + The fully qualified name of a class to use as the + implementation of the scheduler. The default implementation is + org.apache.hadoop.ipc.DefaultRpcScheduler (no-op scheduler) when not using + FairCallQueue. If using FairCallQueue, defaults to + org.apache.hadoop.ipc.DecayRpcScheduler. Use + org.apache.hadoop.ipc.DecayRpcScheduler in conjunction with the Fair Call + Queue. + + + + + ipc.[port_number].scheduler.priority.levels + 4 + How many priority levels to use within the scheduler and call + queue. This property applies to RpcScheduler and CallQueue. + + + + + ipc.[port_number].faircallqueue.multiplexer.weights + 8,4,2,1 + How much weight to give to each priority queue. This should be + a comma-separated list of length equal to the number of priority levels. + Weights descend by a factor of 2 (e.g., for 4 levels: 8,4,2,1). + This property applies to WeightedRoundRobinMultiplexer. + + + + + ipc.[port_number].identity-provider.impl + org.apache.hadoop.ipc.UserIdentityProvider + The identity provider mapping user requests to their identity. + This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].cost-provider.impl + org.apache.hadoop.ipc.DefaultCostProvider + The cost provider mapping user requests to their cost. To + enable determination of cost based on processing time, use + org.apache.hadoop.ipc.WeightedTimeCostProvider. + This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].decay-scheduler.period-ms + 5000 + How frequently the decay factor should be applied to the + operation counts of users. Higher values have less overhead, but respond + less quickly to changes in client behavior. + This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].decay-scheduler.decay-factor + 0.5 + When decaying the operation counts of users, the multiplicative + decay factor to apply. Higher values will weight older operations more + strongly, essentially giving the scheduler a longer memory, and penalizing + heavy clients for a longer period of time. + This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].decay-scheduler.thresholds + 13,25,50 + The client load threshold, as an integer percentage, for each + priority queue. Clients producing less load, as a percent of total + operations, than specified at position i will be given priority i. This + should be a comma-separated list of length equal to the number of priority + levels minus 1 (the last is implicitly 100). + Thresholds ascend by a factor of 2 (e.g., for 4 levels: 13,25,50). + This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].decay-scheduler.backoff.responsetime.enable + false + Whether or not to enable the backoff by response time feature. + This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].decay-scheduler.backoff.responsetime.thresholds + 10s,20s,30s,40s + The response time thresholds, as time durations, for each + priority queue. If the average response time for a queue is above this + threshold, backoff will occur in lower priority queues. This should be a + comma-separated list of length equal to the number of priority levels. + Threshold increases by 10s per level (e.g., for 4 levels: 10s,20s,30s,40s) + This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].decay-scheduler.metrics.top.user.count + 10 + The number of top (i.e., heaviest) users to emit metric + information about. This property applies to DecayRpcScheduler. + + + + + ipc.[port_number].weighted-cost.lockshared + 10 + The weight multiplier to apply to the time spent in the + processing phase which holds a shared (read) lock. + This property applies to WeightedTimeCostProvider. + + + + + ipc.[port_number].weighted-cost.lockexclusive + 100 + The weight multiplier to apply to the time spent in the + processing phase which holds an exclusive (write) lock. + This property applies to WeightedTimeCostProvider. + + + + + ipc.[port_number].weighted-cost.handler + 1 + The weight multiplier to apply to the time spent in the + HANDLER phase which do not involve holding a lock. + See org.apache.hadoop.ipc.ProcessingDetails.Timing for more details on + this phase. This property applies to WeightedTimeCostProvider. + + + + + ipc.[port_number].weighted-cost.lockfree + 1 + The weight multiplier to apply to the time spent in the + LOCKFREE phase which do not involve holding a lock. + See org.apache.hadoop.ipc.ProcessingDetails.Timing for more details on + this phase. This property applies to WeightedTimeCostProvider. + + + + + ipc.[port_number].weighted-cost.response + 1 + The weight multiplier to apply to the time spent in the + RESPONSE phase which do not involve holding a lock. + See org.apache.hadoop.ipc.ProcessingDetails.Timing for more details on + this phase. This property applies to WeightedTimeCostProvider. + + +