Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-9812. Streamer threads leak if failure happens when closing DFSOutputStream. Contributed by Lin Yiqun. 

(cherry picked from commit 352d299cf8)
(cherry picked from commit fe0009a2bd)
This commit is contained in:
Akira Ajisaka 2016-03-08 10:43:17 +09:00
parent e768f8b771
commit 7f43eb9547
2 changed files with 9 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -138,6 +138,9 @@ Release 2.7.3 - UNRELEASED
HDFS-9865. TestBlockReplacement fails intermittently in trunk
(Lin Yiqun via iwasakims)
HDFS-9812. Streamer threads leak if failure happens when closing
DFSOutputStream. (Lin Yiqun via aajisaka)
Release 2.7.2 - 2016-01-25
INCOMPATIBLE CHANGES

8
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
View File

@ -2262,7 +2262,6 @@ public class DFSOutputStream extends FSOutputSummer
flushInternal(); // flush all data to Datanodes
// get last block before destroying the streamer
ExtendedBlock lastBlock = streamer.getBlock();
closeThreads(false);
TraceScope scope = Trace.startSpan("completeFile", Sampler.NEVER);
try {
completeFile(lastBlock);
@ -2271,7 +2270,12 @@ public class DFSOutputStream extends FSOutputSummer
}
} catch (ClosedChannelException e) {
} finally {
setClosed();
// Failures may happen when flushing data.
// Streamers may keep waiting for the new block information.
// Thus need to force closing these threads.
// Don't need to call setClosed() because closeThreads(true)
// calls setClosed() in the finally block.
closeThreads(true);
}
}