From dcad99892d356cabe21874d4c11fcc719d6b78e8 Mon Sep 17 00:00:00 2001 From: Varun Vasudev Date: Thu, 31 Mar 2016 14:05:49 +0530 Subject: [PATCH] YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki. (cherry picked from commit 0064cba169d1bb761f6e81ee86830be598d7c500) --- .../src/main/resources/yarn-default.xml | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml index ea1afe48ae0..33cd9193f0e 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml @@ -908,6 +908,64 @@ 600000 + + + If true, run the policy but do not affect the cluster with preemption and kill events. + + yarn.resourcemanager.monitor.capacity.preemption.observe_only + false + + + + + Time in milliseconds between invocations of this ProportionalCapacityPreemptionPolicy + policy. + + yarn.resourcemanager.monitor.capacity.preemption.monitoring_interval + 3000 + + + + + Time in milliseconds between requesting a preemption from an application and killing + the container. + + yarn.resourcemanager.monitor.capacity.preemption.max_wait_before_kill + 15000 + + + + + Maximum percentage of resources preempted in a single round. By controlling this valueone + can throttle the pace at which containers are reclaimed from the cluster. After computing + the total desired preemption, the policy scales it back within this limit. + + yarn.resourcemanager.monitor.capacity.preemption.total_preemption_per_round + 0.1 + + + + + Maximum amount of resources above the target capacity ignored for preemption. + This defines a deadzone around the target capacity that helps prevent thrashing and + oscillations around the computed target balance. High values would slow the time to capacity + and (absent natural.completions) it might prevent convergence to guaranteed capacity. + + yarn.resourcemanager.monitor.capacity.preemption.max_ignored_over_capacity + 0.1 + + + + + Given a computed preemption target, account for containers naturally expiring and preempt + only this percentage of the delta. This determines the rate of geometric convergence into + the deadzone (MAX_IGNORED_OVER_CAPACITY). For example, a termination factor of 0.5 will reclaim + almost 95% of resources within 5 * #WAIT_TIME_BEFORE_KILL, even absent natural termination. + + yarn.resourcemanager.monitor.capacity.preemption.natural_termination_factor + 0.2 + +