Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

YARN-7665. Allow FS scheduler state dump to be turned on/off separately from FS debug log. (Wilfred Spiegelenburg via Haibo Chen)

This commit is contained in:
Haibo Chen 2018-01-09 16:45:35 -08:00
parent 47563d86fe
commit 8ee7080e5d
2 changed files with 7 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
View File

@ -307,10 +307,12 @@ public class FairScheduler extends
private void dumpSchedulerState() {
FSQueue rootQueue = queueMgr.getRootQueue();
Resource clusterResource = getClusterResource();
LOG.debug("FairScheduler state: Cluster Capacity: " + clusterResource +
STATE_DUMP_LOG.debug(
"FairScheduler state: Cluster Capacity: " + clusterResource +
" Allocations: " + rootMetrics.getAllocatedResources() +
" Availability: " + Resource.newInstance(
rootMetrics.getAvailableMB(), rootMetrics.getAvailableVirtualCores()) +
rootMetrics.getAvailableMB(),
rootMetrics.getAvailableVirtualCores()) +
" Demand: " + rootQueue.getDemand());
STATE_DUMP_LOG.debug(rootQueue.dumpState());
@ -351,7 +353,7 @@ public class FairScheduler extends
}
// Log debug information
if (LOG.isDebugEnabled()) {
if (STATE_DUMP_LOG.isDebugEnabled()) {
if (--updatesToSkipForDebug < 0) {
updatesToSkipForDebug = UPDATE_DEBUG_FREQUENCY;
dumpSchedulerState();

4
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
View File

@ -264,6 +264,6 @@ When an application is moved to a queue, its existing allocations become counted
###Dumping Fair Scheduler state
Fair Scheduler is able to dump its state periodically. It is disabled by default. The administrator can enable it by setting org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler logging level to DEBUG.
Fair Scheduler is able to dump its state periodically. It is disabled by default. The administrator can enable it by setting `org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump` logging level to DEBUG.
Fair Scheduler logs go to the Resource Manager log file by default. Fair Scheduler state dumps can potentially generate large amount of log data. Uncomment the "Fair scheduler state dump" section in log4j.properties to dump the state into a separate file.
Fair Scheduler logs go to the Resource Manager log file by default. Fair Scheduler state dumps can potentially generate a large amount of log data. Uncomment the "Fair scheduler state dump" section in log4j.properties to dump the state into a separate file.