Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-16910. Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN (#5359)

This commit is contained in:
huhaiyang 2023-02-09 10:47:57 +08:00 committed by GitHub
parent 4fcceff535
commit d5c046518e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java
View File

@ -84,9 +84,9 @@ public class EditLogFileOutputStream extends EditLogOutputStream {
doubleBuf = new EditsDoubleBuffer(size); doubleBuf = new EditsDoubleBuffer(size);
RandomAccessFile rp; RandomAccessFile rp;
if (shouldSyncWritesAndSkipFsync) { if (shouldSyncWritesAndSkipFsync) {
rp = new RandomAccessFile(name, "rw"); rp = new RandomAccessFile(name, "rwd");
} else { } else {
rp = new RandomAccessFile(name, "rws"); rp = new RandomAccessFile(name, "rw");
} }
try { try {
fp = new FileOutputStream(rp.getFD()); // open for append fp = new FileOutputStream(rp.getFD()); // open for append

4
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
View File

@ -2721,10 +2721,10 @@
<description> <description>
Specifies whether to flush edit log file channel. When set, expensive Specifies whether to flush edit log file channel. When set, expensive
FileChannel#force calls are skipped and synchronous disk writes are FileChannel#force calls are skipped and synchronous disk writes are
enabled instead by opening the edit log file with RandomAccessFile("rws") enabled instead by opening the edit log file with RandomAccessFile("rwd")
flags. This can significantly improve the performance of edit log writes flags. This can significantly improve the performance of edit log writes
on the Windows platform. on the Windows platform.
Note that the behavior of the "rws" flags is platform and hardware specific Note that the behavior of the "rwd" flags is platform and hardware specific
and might not provide the same level of guarantees as FileChannel#force. and might not provide the same level of guarantees as FileChannel#force.
For example, the write will skip the disk-cache on SAS and SCSI devices For example, the write will skip the disk-cache on SAS and SCSI devices
while it might not on SATA devices. This is an expert level setting, while it might not on SATA devices. This is an expert level setting,