Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-27166 WAL value compression minor improvements (#4584) 

A larger IO buffer for absorbing WALCodec writes can improve the compression
ratio of larger values, because the compressor will be given a larger internal
buffer over which there will be more match opportunities. Does not impact the
ability to read existing written files.

Also, reset the BAOS internal buffer on the way out of compress() so potential
large-ish buffers do not linger on the heap longer than necessary.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
This commit is contained in:
Andrew Purtell 2022-06-28 12:49:25 -07:00
parent bd08ebb21c
commit e4d2895cad
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
View File

@ -71,7 +71,7 @@ public class CompressionContext {
*/
static class ValueCompressor {
static final int IO_BUFFER_SIZE = 4096;
static final int IO_BUFFER_SIZE = 64 * 1024; // bigger buffer improves large edit compress ratio
private final Compression.Algorithm algorithm;
private Compressor compressor;
@ -97,12 +97,12 @@ public class CompressionContext {
compressor = algorithm.getCompressor();
}
compressedOut = algorithm.createCompressionStream(lowerOut, compressor, IO_BUFFER_SIZE);
} else {
lowerOut.reset();
}
compressedOut.write(valueArray, valueOffset, valueLength);
compressedOut.flush();
return lowerOut.toByteArray();
final byte[] compressed = lowerOut.toByteArray();
lowerOut.reset(); // Reset now to minimize the overhead of keeping around the BAOS
return compressed;
}
public int decompress(InputStream in, int inLength, byte[] outArray, int outOffset,