HBASE-4143 HTable.doPut(List) should check the writebuffer length every so often

(Doug Meil via Ted Yu)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1151689 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Zhihong Yu 2011-07-28 00:24:11 +00:00
parent 5148146daf
commit 6c838dc0ef
3 changed files with 17 additions and 3 deletions

View File

@ -340,6 +340,8 @@ Release 0.91.0 - Unreleased
HBASE-4139 [stargate] Update ScannerModel with support for filter package
additions
HBASE-1938 Make in-memory table scanning faster (nkeywal)
HBASE-4143 HTable.doPut(List) should check the writebuffer length every so often
(Doug Meil via Ted Yu)
TASKS
HBASE-3559 Move report of split to master OFF the heartbeat channel

View File

@ -113,7 +113,8 @@ public class HTable implements HTableInterface, Closeable {
private long maxScannerResultSize;
private boolean closed;
private int operationTimeout;
private static final int DOPUT_WB_CHECK = 10; // i.e., doPut checks the writebuffer every X Puts.
/**
* Creates an object to access a HBase table.
* Internally it creates a new instance of {@link Configuration} and a new
@ -706,10 +707,20 @@ public class HTable implements HTableInterface, Closeable {
}
private void doPut(final List<Put> puts) throws IOException {
int n = 0;
for (Put put : puts) {
validatePut(put);
writeBuffer.add(put);
currentWriteBufferSize += put.heapSize();
// we need to periodically see if the writebuffer is full instead of waiting until the end of the List
n++;
if (n == DOPUT_WB_CHECK) {
if (autoFlush || currentWriteBufferSize > writeBufferSize) {
flushCommits();
n = 0;
}
}
}
if (autoFlush || currentWriteBufferSize > writeBufferSize) {
flushCommits();

View File

@ -190,8 +190,9 @@ public interface HTableInterface {
* until the internal buffer is full.
* <p>
* This can be used for group commit, or for submitting user defined
* batches, but sending large lists of values is not recommended. That may
* produce performance problems.
* batches. The writeBuffer will be periodically inspected while the List
* is processed, so depending on the List size the writeBuffer may flush
* not at all, or more than once.
* @param puts The list of mutations to apply. The batch put is done by
* aggregating the iteration of the Puts over the write buffer
* at the client-side for a single RPC call.