Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-27166 WAL value compression minor improvements (#4584) 

A larger IO buffer for absorbing WALCodec writes can improve the compression
ratio of larger values, because the compressor will be given a larger internal
buffer over which there will be more match opportunities. Does not impact the
ability to read existing written files.

Also, reset the BAOS internal buffer on the way out of compress() so potential
large-ish buffers do not linger on the heap longer than necessary.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
This commit is contained in:
Andrew Purtell 2022-06-28 12:49:25 -07:00 committed by GitHub
parent d7f6861e8d
commit ca34bdccc3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
View File

@ -71,7 +71,7 @@ public class CompressionContext {
*/
static class ValueCompressor {
static final int IO_BUFFER_SIZE = 4096;
static final int IO_BUFFER_SIZE = 64 * 1024; // bigger buffer improves large edit compress ratio
private final Compression.Algorithm algorithm;
private Compressor compressor;
@ -97,12 +97,12 @@ public class CompressionContext {
compressor = algorithm.getCompressor();
}
compressedOut = algorithm.createCompressionStream(lowerOut, compressor, IO_BUFFER_SIZE);
} else {
lowerOut.reset();
}
compressedOut.write(valueArray, valueOffset, valueLength);
compressedOut.flush();
return lowerOut.toByteArray();
final byte[] compressed = lowerOut.toByteArray();
lowerOut.reset(); // Reset now to minimize the overhead of keeping around the BAOS
return compressed;
}
public int decompress(InputStream in, int inLength, byte[] outArray, int outOffset,