Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw)

Browse Source 
(cherry picked from commit fd57ab2002f97dcc83d455a5e0c770c8efde77a4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
This commit is contained in:
Jason Lowe 2015-02-05 16:40:09 +00:00
parent 4cd9657b71
commit 2dc4af2b93
2 changed files with 10 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-mapreduce-project/CHANGES.txt
View File

@ -48,6 +48,8 @@ Release 2.7.0 - UNRELEASED
from key and value at the end of the iteration to save memory.
(Zhihai Xu via kasha)
MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw)
BUG FIXES
MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationId()

8
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
View File

@ -263,6 +263,10 @@ public Collection<HistoryFileInfo> values() {
public HistoryFileInfo get(JobId jobId) {
return cache.get(jobId);
}
public boolean isFull() {
return cache.size() >= maxSize;
}
}
/**
@ -668,6 +672,10 @@ void initExisting() throws IOException {
for (FileStatus fs : timestampedDirList) {
// TODO Could verify the correct format for these directories.
addDirectoryToSerialNumberIndex(fs.getPath());
}
for (int i= timestampedDirList.size() - 1;
i >= 0 && !jobListCache.isFull(); i--) {
FileStatus fs = timestampedDirList.get(i);
addDirectoryToJobListCache(fs.getPath());
}
}