Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

YARN-6028. Add document for container metrics (Contributed by Weiwei Yang via Daniel Templeton) 

(cherry picked from commit f5839fd6df)
This commit is contained in:
Daniel Templeton 2017-01-19 12:45:17 -08:00
parent bed700e98f
commit be7e4a0805
1 changed files with 51 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
View File

@ -443,6 +443,57 @@ NodeManagerMetrics shows the statistics of the containers in the node. Each metr
| `goodLocalDirsDiskUtilizationPerc` | Current disk utilization percentage across all good local directories |
| `goodLogDirsDiskUtilizationPerc` | Current disk utilization percentage across all good log directories |
ContainerMetrics
------------------
ContainerMetrics shows the resource utilization statistics of a container. Each metrics record contains tags such as ContainerPid and Hostname as additional information along with metrics.
| Name | Description |
|:---- |:---- |
| `pMemLimitMBs` | Physical memory limit of the container in MB |
| `vMemLimitMBs` | Virtual memory limit of the container in MB |
| `vCoreLimit` | CPU limit of the container in number of vcores |
| `launchDurationMs` | Container launch duration in msec |
| `localizationDurationMs` | Container localization duration in msec |
| `StartTime` | Time in msec when container starts |
| `FinishTime` | Time in msec when container finishes |
| `ExitCode` | Container's exit code |
| `PMemUsageMBsNumUsage` | Total number of physical memory used metrics |
| `PMemUsageMBsAvgMBs` | Average physical memory used in MB |
| `PMemUsageMBsStdevMBs` | Standard deviation of the physical memory used in MB |
| `PMemUsageMBsMinMBs` | Minimum physical memory used in MB |
| `PMemUsageMBsMaxMBs` | Maximum physical memory used in MB |
| `PMemUsageMBsIMinMBs` | Minimum physical memory used in MB of current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `PMemUsageMBsIMaxMBs` | Maximum physical memory used in MB of current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `PMemUsageMBsINumUsage` | Total number of physical memory used metrics in current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `PCpuUsagePercentNumUsage` | Total number of physical CPU cores percent used metrics |
| `PCpuUsagePercentAvgPercents` | Average physical CPU cores percent used |
| `PCpuUsagePercentStdevPercents` | Standard deviation of physical CPU cores percent used |
| `PCpuUsagePercentMinPercents` | Minimum physical CPU cores percent used|
| `PCpuUsagePercentMaxPercents` | Maximum physical CPU cores percent used |
| `PCpuUsagePercentIMinPercents` | Minimum physical CPU cores percent used in current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `PCpuUsagePercentIMaxPercents` | Maximum physical CPU cores percent used in current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `PCpuUsagePercentINumUsage` | Total number of physical CPU cores used metrics in current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `MilliVcoreUsageNumUsage` | Total number of vcores used metrics |
| `MilliVcoreUsageAvgMilliVcores` | 1000 times the average vcores used |
| `MilliVcoreUsageStdevMilliVcores` | 1000 times the standard deviation of vcores used |
| `MilliVcoreUsageMinMilliVcores` | 1000 times the minimum vcores used |
| `MilliVcoreUsageMaxMilliVcores` | 1000 times the maximum vcores used |
| `MilliVcoreUsageIMinMilliVcores` | 1000 times the average vcores used in current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `MilliVcoreUsageIMaxMilliVcores` | 1000 times the maximum vcores used in current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `MilliVcoreUsageINumUsage` | Total number of vcores used metrics in current _interval_ (the time of _interval_ is specified by yarn.nodemanager.container-metrics.period-ms) |
| `PMemUsageMBHistogramNumUsage` | Total number of physical memory used metrics (1 second granularity) |
| `PMemUsageMBHistogram50thPercentileMBs` | The 50th percentile of physical memory used in MB (1 second granularity) |
| `PMemUsageMBHistogram75thPercentileMBs` | The 75th percentile of physical memory used in MB (1 second granularity) |
| `PMemUsageMBHistogram90thPercentileMBs` | The 90th percentile of physical memory used in MB (1 second granularity) |
| `PMemUsageMBHistogram95thPercentileMBs` | The 95th percentile of physical memory used in MB (1 second granularity) |
| `PMemUsageMBHistogram99thPercentileMBs` | The 99th percentile of physical memory used in MB (1 second granularity) |
| `PCpuUsagePercentHistogramNumUsage` | Total number of physical CPU cores used metrics (1 second granularity) |
| `PCpuUsagePercentHistogram50thPercentilePercents` | The 50th percentile of physical CPU cores percent used (1 second granularity) |
| `PCpuUsagePercentHistogram75thPercentilePercents` | The 75th percentile of physical CPU cores percent used (1 second granularity) |
| `PCpuUsagePercentHistogram90thPercentilePercents` | The 90th percentile of physical CPU cores percent used (1 second granularity) |
| `PCpuUsagePercentHistogram95thPercentilePercents` | The 95th percentile of physical CPU cores percent used (1 second granularity) |
| `PCpuUsagePercentHistogram99thPercentilePercents` | The 99th percentile of physical CPU cores percent used (1 second granularity) |
ugi context
===========
@ -507,5 +558,3 @@ StartupProgress metrics shows the statistics of NameNode startup. Four metrics a
| *phase*`ElapsedTime` | Total elapsed time in the phase in milliseconds |
| *phase*`Total` | Total number of steps in the phase |
| *phase*`PercentComplete` | Current rate completed in the phase  (The max value is not 100 but 1.0) |