Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

MAPREDUCE-4087. [Gridmix] GenerateDistCacheData job of Gridmix can become slow in some cases (ravigummadi via tgraves). 

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1461243 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Thomas Graves 2013-03-26 18:00:30 +00:00
parent 288393399a
commit 57f91f9541
2 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-mapreduce-project/CHANGES.txt
View File

@ -87,6 +87,9 @@ Release 2.0.5-beta - UNRELEASED
job if the trace contains null taskStatus for a task. (ravigummadi via job if the trace contains null taskStatus for a task. (ravigummadi via
tgraves) tgraves)
MAPREDUCE-4087. [Gridmix] GenerateDistCacheData job of Gridmix can
become slow in some cases (ravigummadi via tgraves).
Release 2.0.4-alpha - UNRELEASED Release 2.0.4-alpha - UNRELEASED

7
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GenerateDistCacheData.java
View File

@ -156,10 +156,11 @@ class GenerateDistCacheData extends GridmixJob {
FSDataOutputStream dos = FSDataOutputStream dos =
FileSystem.create(fs, path, new FsPermission((short)0755)); FileSystem.create(fs, path, new FsPermission((short)0755));
for (long bytes = key.get(); bytes > 0; bytes -= val.getLength()) { int size = 0;
for (long bytes = key.get(); bytes > 0; bytes -= size) {
r.nextBytes(val.getBytes()); r.nextBytes(val.getBytes());
val.setSize((int)Math.min(val.getLength(), bytes)); size = (int)Math.min(val.getLength(), bytes);
dos.write(val.getBytes(), 0, val.getLength());// Write to distCache file dos.write(val.getBytes(), 0, size);// Write to distCache file
} }
dos.close(); dos.close();
} }