From faa38e1aa49907254bf981662a8aeb5dc52e75e3 Mon Sep 17 00:00:00 2001 From: Tsuyoshi Ozawa Date: Tue, 1 Sep 2015 17:54:14 +0900 Subject: [PATCH] YARN-2801. Add documentation for node labels feature. Contributed by Wangda Tan and Naganarasimha G R. --- hadoop-project/src/site/site.xml | 1 + hadoop-yarn-project/CHANGES.txt | 3 + .../src/site/markdown/NodeLabel.md | 140 ++++++++++++++++++ 3 files changed, 144 insertions(+) create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeLabel.md diff --git a/hadoop-project/src/site/site.xml b/hadoop-project/src/site/site.xml index ee0dfcdc160..272ea7bbf30 100644 --- a/hadoop-project/src/site/site.xml +++ b/hadoop-project/src/site/site.xml @@ -121,6 +121,7 @@ + diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt index 999654dab50..e2b13078f4a 100644 --- a/hadoop-yarn-project/CHANGES.txt +++ b/hadoop-yarn-project/CHANGES.txt @@ -827,6 +827,9 @@ Release 2.7.2 - UNRELEASED YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. (Xuan Gong via jianhe) + YARN-2801. Add documentation for node labels feature. (Wangda Tan and Naganarasimha + G R via ozawa) + OPTIMIZATIONS BUG FIXES diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeLabel.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeLabel.md new file mode 100644 index 00000000000..87019cddbae --- /dev/null +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeLabel.md @@ -0,0 +1,140 @@ + + +YARN Node Labels +=============== + +# Overview +Node label is a way to group nodes with similar characteristics and applications can specify where to run. + +Now we only support node partition, which is: + +* One node can have only one node partition, so a cluster is partitioned to several disjoint sub-clusters by node partitions. By default, nodes belong to DEFAULT partition (partition="") +* User need to configure how much resources of each partition can be used by different queues. For more detail, please refer next section. +* There are two kinds of node partitions: + * Exclusive: containers will be allocated to nodes with exactly match node partition. (e.g. asking partition=“x” will be allocated to node with partition=“x”, asking DEFAULT partition will be allocated to DEFAULT partition nodes). + * Non-exclusive: if a partition is non-exclusive, it shares idle resource to container requesting DEFAULT partition. + +User can specify set of node labels which can be accessed by each queue, one application can only use subset of node labels that can be accessed by the queue which contains the application. + +# Features +The ```Node Labels``` supports the following features for now: + +* Partition cluster - each node can be assigned one label, so the cluster will be divided to several smaller disjoint partitions. +* ACL of node-labels on queues - user can set accessible node labels on each queue so only some nodes can only be accessed by specific queues. +* Specify percentage of resource of a partition which can be accessed by a queue - user can set percentage like: queue A can access 30% of resources on nodes with label=hbase. Such percentage setting will be consistent with existing resource manager +* Specify required Node Label in resource request, it will only be allocated when node has the same label. If no node label requirement specified, such Resource Request will only be allocated on nodes belong to DEFAULT partition. +* Operability + * Node labels and node labels mapping can be recovered across RM restart + * Update node labels - admin can update labels on nodes and labels on queues + when RM is running + +# Configuration +## Setting up ```ResourceManager``` to enable ```Node Labels```: + +Setup following properties in ```yarn-site.xml``` + +Property | Value +--- | ---- +yarn.node-labels.fs-store.root-dir | hdfs://namenode:port/path/to/store/node-labels/ +yarn.node-labels.enabled | true + +Notes: + +* Make sure ```yarn.node-labels.fs-store.root-dir``` is created and ```ResourceManager``` has permission to access it. (Typically from “yarn” user) +* If user want to store node label to local file system of RM (instead of HDFS), paths like `file:///home/yarn/node-label` can be used + +### Add/modify node labels list and node-to-labels mapping to YARN +* Add cluster node labels list: + * Executing ```yarn rmadmin -addToClusterNodeLabels "label_1(exclusive=true/false),label_2(exclusive=true/false)"``` to add node label. + * If user don’t specify “(exclusive=…)”, execlusive will be ```true``` by default. + * Run ```yarn cluster --list-node-labels``` to check added node labels are visible in the cluster. + +* Add labels to nodes + * Executing ```yarn rmadmin -replaceLabelsOnNode “node1[:port]=label1 node2=label2”```. Added label1 to node1, label2 to node2. If user don’t specify port, it added the label to all ```NodeManagers``` running on the node. + +## Configuration of Schedulers for node labels +### Capacity Scheduler Configuration +Property | Value +----- | ------ +yarn.scheduler.capacity.``.capacity | Set the percentage of the queue can access to nodes belong to DEFAULT partition. The sum of DEFAULT capacities for direct children under each parent, must be equal to 100. +yarn.scheduler.capacity.``.accessible-node-labels | Admin need specify labels can be accessible by each queue, split by comma, like “hbase,storm” means queue can access label hbase and storm. All queues can access to nodes without label, user don’t have to specify that. If user don’t specify this field, it will inherit from its parent. If user want to explicitly specify a queue can only access nodes without labels, just put a space as the value. +yarn.scheduler.capacity.``.accessible-node-labels.`