YARN-4098. Document ApplicationPriority feature. Contributed by Rohith Sharma K S
This commit is contained in:
parent
5273413411
commit
ad997fab67
|
@ -660,6 +660,8 @@ Release 2.8.0 - UNRELEASED
|
||||||
YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports
|
YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports
|
||||||
information. (Sunil G via wangda)
|
information. (Sunil G via wangda)
|
||||||
|
|
||||||
|
YARN-4098. Document ApplicationPriority feature. (Rohith Sharma K S via jianhe)
|
||||||
|
|
||||||
OPTIMIZATIONS
|
OPTIMIZATIONS
|
||||||
|
|
||||||
YARN-3339. TestDockerContainerExecutor should pull a single image and not
|
YARN-3339. TestDockerContainerExecutor should pull a single image and not
|
||||||
|
|
|
@ -24,6 +24,7 @@ Hadoop: Capacity Scheduler
|
||||||
* [Queue Properties](#Queue_Properties)
|
* [Queue Properties](#Queue_Properties)
|
||||||
* [Other Properties](#Other_Properties)
|
* [Other Properties](#Other_Properties)
|
||||||
* [Reviewing the configuration of the CapacityScheduler](#Reviewing_the_configuration_of_the_CapacityScheduler)
|
* [Reviewing the configuration of the CapacityScheduler](#Reviewing_the_configuration_of_the_CapacityScheduler)
|
||||||
|
* [Setup for application priority](#Setup_for_application_priority.)
|
||||||
* [Changing Queue Configuration](#Changing_Queue_Configuration)
|
* [Changing Queue Configuration](#Changing_Queue_Configuration)
|
||||||
|
|
||||||
Purpose
|
Purpose
|
||||||
|
@ -71,6 +72,8 @@ The `CapacityScheduler` supports the following features:
|
||||||
|
|
||||||
* **Queue Mapping based on User or Group** - This feature allows users to map a job to a specific queue based on the user or group.
|
* **Queue Mapping based on User or Group** - This feature allows users to map a job to a specific queue based on the user or group.
|
||||||
|
|
||||||
|
* **Priority Scheduling** - This feature allows applications to be submitted and scheduled with different priorities. Higher integer value indicates higher priority for an application. Currently Application priority is supported only for FIFO ordering policy.
|
||||||
|
|
||||||
Configuration
|
Configuration
|
||||||
-------------
|
-------------
|
||||||
|
|
||||||
|
@ -201,6 +204,28 @@ Example:
|
||||||
|
|
||||||
* The */scheduler* web-page should show the resource usages of individual queues.
|
* The */scheduler* web-page should show the resource usages of individual queues.
|
||||||
|
|
||||||
|
###Setup for application priority.
|
||||||
|
|
||||||
|
Application priority works only along with FIFO ordering policy. Default ordering policy is FIFO.
|
||||||
|
|
||||||
|
Default priority for an application can be at cluster level and queue level.
|
||||||
|
|
||||||
|
* Cluster-level priority : Any application that is submitted with a priority is greater then cluster max priority then application priority will be reset with cluster-max priority.
|
||||||
|
$HADOOP_HOME/etc/hadoop/yarn-site.xml is the configuration file for cluster-max priority.
|
||||||
|
|
||||||
|
| Property | Description |
|
||||||
|
|:---- |:---- |
|
||||||
|
| `yarn.cluster.max-application-priority` | Defines maximum application priority in a cluster. |
|
||||||
|
|
||||||
|
* Leaf Queue-level priority : Each leaf queue provides default priority by the administrator. If application is not submitted with any priority, then default priority of the queue will be taken in account.
|
||||||
|
$HADOOP_HOME/etc/hadoop/capacity-scheduler.xml is the configuration file for queue-level priority.
|
||||||
|
|
||||||
|
| Property | Description |
|
||||||
|
|:---- |:---- |
|
||||||
|
| `yarn.scheduler.capacity.root.<leaf-queue-path>.default-application-priority` | Defines default application priority in a leaf queue. |
|
||||||
|
|
||||||
|
**Note:** Priority of an application will not be changed when application is moved to different queue.
|
||||||
|
|
||||||
Changing Queue Configuration
|
Changing Queue Configuration
|
||||||
----------------------------
|
----------------------------
|
||||||
|
|
||||||
|
|
|
@ -67,11 +67,13 @@ Usage: `yarn application [options] `
|
||||||
|
|
||||||
| COMMAND\_OPTIONS | Description |
|
| COMMAND\_OPTIONS | Description |
|
||||||
|:---- |:---- |
|
|:---- |:---- |
|
||||||
|
| -appId \<ApplicationId\> | Specify Application Id to be operated |
|
||||||
| -appStates \<States\> | Works with -list to filter applications based on input comma-separated list of application states. The valid application state can be one of the following: ALL, NEW, NEW\_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED |
|
| -appStates \<States\> | Works with -list to filter applications based on input comma-separated list of application states. The valid application state can be one of the following: ALL, NEW, NEW\_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED |
|
||||||
| -appTypes \<Types\> | Works with -list to filter applications based on input comma-separated list of application types. |
|
| -appTypes \<Types\> | Works with -list to filter applications based on input comma-separated list of application types. |
|
||||||
| -list | Lists applications from the RM. Supports optional use of -appTypes to filter applications based on application type, and -appStates to filter applications based on application state. |
|
| -list | Lists applications from the RM. Supports optional use of -appTypes to filter applications based on application type, and -appStates to filter applications based on application state. |
|
||||||
| -kill \<ApplicationId\> | Kills the application. |
|
| -kill \<ApplicationId\> | Kills the application. |
|
||||||
| -status \<ApplicationId\> | Prints the status of the application. |
|
| -status \<ApplicationId\> | Prints the status of the application. |
|
||||||
|
| -updatePriority \<Priority\> | Update priority of an application. ApplicationId can be passed using 'appId' option. |
|
||||||
|
|
||||||
Prints application(s) report/kill application
|
Prints application(s) report/kill application
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue