From 5e602d10d99b30285aac45645372e48137af8d70 Mon Sep 17 00:00:00 2001 From: Sanford Ryza Date: Wed, 9 Oct 2013 05:57:08 +0000 Subject: [PATCH] YARN-976. Document the meaning of a virtual core. (Sandy Ryza) git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530501 13f79535-47bb-0310-9956-ffa450edef68 --- hadoop-yarn-project/CHANGES.txt | 2 ++ .../hadoop/yarn/api/records/Resource.java | 33 ++++++++++++------- 2 files changed, 23 insertions(+), 12 deletions(-) diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt index ab99d670283..4aca0e923e5 100644 --- a/hadoop-yarn-project/CHANGES.txt +++ b/hadoop-yarn-project/CHANGES.txt @@ -42,6 +42,8 @@ Release 2.3.0 - UNRELEASED YARN-465. fix coverage org.apache.hadoop.yarn.server.webproxy (Aleksey Gorshkov and Andrey Klochkov via jlowe) + YARN-976. Document the meaning of a virtual core. (Sandy Ryza) + OPTIMIZATIONS BUG FIXES diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java index 5a85e7ce34a..88b57f1c68f 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java @@ -28,7 +28,18 @@ *

Resource models a set of computer resources in the * cluster.

* - *

Currrently it only models memory.

+ *

Currently it models both memory and CPU.

+ * + *

The unit for memory is megabytes. CPU is modeled with virtual cores + * (vcores), a unit for expressing parallelism. A node's capacity should + * be configured with virtual cores equal to its number of physical cores. A + * container should be requested with the number of cores it can saturate, i.e. + * the average number of threads it expects to have runnable at a time.

+ * + *

Virtual cores take integer values and thus currently CPU-scheduling is + * very coarse. A complementary axis for CPU requests that represents processing + * power will likely be added in the future to enable finer-grained resource + * configuration.

* *

Typically, applications request Resource of suitable * capability to run their component tasks.

@@ -69,11 +80,10 @@ public static Resource newInstance(int memory, int vCores) { /** * Get number of virtual cpu cores of the resource. * - * We refer to virtual cores to clarify that these represent - * normalized cores which may have a m:n relationship w.r.t - * physical cores available on the compute nodes. Furthermore, they also - * represent idealized cores since the cluster might be composed - * of heterogenous nodes. + * Virtual cores are a unit for expressing CPU parallelism. A node's capacity + * should be configured with virtual cores equal to its number of physical cores. + * A container should be requested with the number of cores it can saturate, i.e. + * the average number of threads it expects to have runnable at a time. * * @return num of virtual cpu cores of the resource */ @@ -84,12 +94,11 @@ public static Resource newInstance(int memory, int vCores) { /** * Set number of virtual cpu cores of the resource. * - * We refer to virtual cores to clarify that these represent - * normalized cores which may have a m:n relationship w.r.t - * physical cores available on the compute nodes. Furthermore, they also - * represent idealized cores since the cluster might be composed - * of heterogenous nodes. - * + * Virtual cores are a unit for expressing CPU parallelism. A node's capacity + * should be configured with virtual cores equal to its number of physical cores. + * A container should be requested with the number of cores it can saturate, i.e. + * the average number of threads it expects to have runnable at a time. + * * @param vCores number of virtual cpu cores of the resource */ @Public