diff --git a/hadoop-common-project/hadoop-common/src/site/apt/Metrics.apt.vm b/hadoop-common-project/hadoop-common/src/site/apt/Metrics.apt.vm index 14cc712f0e4..02ff28bc644 100644 --- a/hadoop-common-project/hadoop-common/src/site/apt/Metrics.apt.vm +++ b/hadoop-common-project/hadoop-common/src/site/apt/Metrics.apt.vm @@ -605,6 +605,145 @@ dfs context | packets in nanoseconds *-------------------------------------+--------------------------------------+ +yarn context + +* ClusterMetrics + + ClusterMetrics shows the metrics of the YARN cluster from the + ResourceManager's perspective. Each metrics record contains + Hostname tag as additional information along with metrics. + +*-------------------------------------+--------------------------------------+ +|| Name || Description +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of active NodeManagers +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of decommissioned NodeManagers +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of lost NodeManagers for not sending + | heartbeats +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of unhealthy NodeManagers +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of rebooted NodeManagers +*-------------------------------------+--------------------------------------+ + +* QueueMetrics + + QueueMetrics shows an application queue from the + ResourceManager's perspective. Each metrics record shows + the statistics of each queue, and contains tags such as + queue name and Hostname as additional information along with metrics. + + In <<>> metrics such as <<>>, you can set the + property <<>> in yarn-site.xml + to change the buckets. The default values is <<<60,300,1440>>>. + +*-------------------------------------+--------------------------------------+ +|| Name || Description +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of running applications whose elapsed time are + | less than 60 minutes +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of running applications whose elapsed time are + | between 60 and 300 minutes +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of running applications whose elapsed time are + | between 300 and 1440 minutes +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of running applications elapsed time are + | more than 1440 minutes +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of submitted applications +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of running applications +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of applications that have not yet been + | assigned by any containers +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of completed applications +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of killed applications +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of failed applications +*-------------------------------------+--------------------------------------+ +|<<>> | Current allocated memory in MB +*-------------------------------------+--------------------------------------+ +|<<>> | Current allocated CPU in virtual cores +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of allocated containers +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of allocated containers +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of released containers +*-------------------------------------+--------------------------------------+ +|<<>> | Current available memory in MB +*-------------------------------------+--------------------------------------+ +|<<>> | Current available CPU in virtual cores +*-------------------------------------+--------------------------------------+ +|<<>> | Current pending memory resource requests in MB that are + | not yet fulfilled by the scheduler +*-------------------------------------+--------------------------------------+ +|<<>> | Current pending CPU allocation requests in virtual + | cores that are not yet fulfilled by the scheduler +*-------------------------------------+--------------------------------------+ +|<<>> | Current pending resource requests that are not + | yet fulfilled by the scheduler +*-------------------------------------+--------------------------------------+ +|<<>> | Current reserved memory in MB +*-------------------------------------+--------------------------------------+ +|<<>> | Current reserved CPU in virtual cores +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of reserved containers +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of active users +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of active applications +*-------------------------------------+--------------------------------------+ +|<<>> | (FairScheduler only) Current fair share of memory in MB +*-------------------------------------+--------------------------------------+ +|<<>> | (FairScheduler only) Current fair share of CPU in + | virtual cores +*-------------------------------------+--------------------------------------+ +|<<>> | (FairScheduler only) Minimum share of memory in MB +*-------------------------------------+--------------------------------------+ +|<<>> | (FairScheduler only) Minimum share of CPU in virtual + | cores +*-------------------------------------+--------------------------------------+ +|<<>> | (FairScheduler only) Maximum share of memory in MB +*-------------------------------------+--------------------------------------+ +|<<>> | (FairScheduler only) Maximum share of CPU in virtual + | cores +*-------------------------------------+--------------------------------------+ + +* NodeManagerMetrics + + NodeManagerMetrics shows the statistics of the containers in the node. + Each metrics record contains Hostname tag as additional information + along with metrics. + +*-------------------------------------+--------------------------------------+ +|| Name || Description +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of launched containers +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of successfully completed containers +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of failed containers +*-------------------------------------+--------------------------------------+ +|<<>> | Total number of killed containers +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of initializing containers +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of running containers +*-------------------------------------+--------------------------------------+ +|<<>> | Current number of allocated containers +*-------------------------------------+--------------------------------------+ +|<<>> | Current allocated memory in GB +*-------------------------------------+--------------------------------------+ +|<<>> | Current available memory in GB +*-------------------------------------+--------------------------------------+ + ugi context * UgiMetrics diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt index e36b937334d..b394a912a35 100644 --- a/hadoop-yarn-project/CHANGES.txt +++ b/hadoop-yarn-project/CHANGES.txt @@ -75,6 +75,9 @@ Release 2.7.0 - UNRELEASED YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes independent of Scheduler type. (Anubhav Dhoot via kasha) + YARN-2157. Added YARN metrics in the documentaion. (Akira AJISAKA via + jianhe) + OPTIMIZATIONS BUG FIXES