Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

YARN-2777. Mark the end of individual log in aggregated log. Contributed 

by Varun Saxena

(cherry picked from commit 1a4b528691)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
This commit is contained in:
Xuan 2015-03-20 10:24:13 -07:00
parent faeb8b9b53
commit b63ff4c36d
3 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-yarn-project/CHANGES.txt
View File

@ -335,6 +335,9 @@ Release 2.7.0 - UNRELEASED
CapacitySchedulerLeafQueueInfo#aMResourceLimit to AMResourceLimit. CapacitySchedulerLeafQueueInfo#aMResourceLimit to AMResourceLimit.
(Rohith via jianhe) (Rohith via jianhe)
YARN-2777. Mark the end of individual log in aggregated log.
(Varun Saxena via xgong)
OPTIMIZATIONS OPTIMIZATIONS
YARN-2990. FairScheduler's delay-scheduling always waits for node-local and YARN-2990. FairScheduler's delay-scheduling always waits for node-local and

1
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
View File

@ -711,6 +711,7 @@ public class AggregatedLogFormat {
pendingRead > buf.length ? buf.length : (int) pendingRead; pendingRead > buf.length ? buf.length : (int) pendingRead;
len = valueStream.read(buf, 0, toRead); len = valueStream.read(buf, 0, toRead);
} }
out.println("End of LogType:" + fileType);
out.println(""); out.println("");
} }

3
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/logaggregation/TestAggregatedLogFormat.java
View File

@ -257,7 +257,8 @@ public class TestAggregatedLogFormat {
+ (logUploadedTime ? ("\nLog Upload Time:" + Times.format(System + (logUploadedTime ? ("\nLog Upload Time:" + Times.format(System
.currentTimeMillis())).length() : 0) .currentTimeMillis())).length() : 0)
+ ("\nLogLength:" + numChars).length() + ("\nLogLength:" + numChars).length()
+ "\nLog Contents:\n".length() + numChars + "\n".length(); + "\nLog Contents:\n".length() + numChars + "\n".length()
+ "End of LogType:stdout\n".length();
Assert.assertTrue("LogType not matched", s.contains("LogType:stdout")); Assert.assertTrue("LogType not matched", s.contains("LogType:stdout"));
Assert.assertTrue("log file:stderr should not be aggregated.", !s.contains("LogType:stderr")); Assert.assertTrue("log file:stderr should not be aggregated.", !s.contains("LogType:stderr"));
Assert.assertTrue("log file:logs should not be aggregated.", !s.contains("LogType:logs")); Assert.assertTrue("log file:logs should not be aggregated.", !s.contains("LogType:logs"));