Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-4143 HTable.doPut(List) should check the writebuffer length every so often 

(Doug Meil via Ted Yu)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1151689 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Zhihong Yu 2011-07-28 00:24:11 +00:00
parent 5148146daf
commit 6c838dc0ef
3 changed files with 17 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGES.txt
View File

@ -340,6 +340,8 @@ Release 0.91.0 - Unreleased
HBASE-4139 [stargate] Update ScannerModel with support for filter package HBASE-4139 [stargate] Update ScannerModel with support for filter package
additions additions
HBASE-1938 Make in-memory table scanning faster (nkeywal) HBASE-1938 Make in-memory table scanning faster (nkeywal)
HBASE-4143 HTable.doPut(List) should check the writebuffer length every so often
(Doug Meil via Ted Yu)
TASKS TASKS
HBASE-3559 Move report of split to master OFF the heartbeat channel HBASE-3559 Move report of split to master OFF the heartbeat channel

11
src/main/java/org/apache/hadoop/hbase/client/HTable.java
View File

@ -113,6 +113,7 @@ public class HTable implements HTableInterface, Closeable {
private long maxScannerResultSize; private long maxScannerResultSize;
private boolean closed; private boolean closed;
private int operationTimeout; private int operationTimeout;
private static final int DOPUT_WB_CHECK = 10; // i.e., doPut checks the writebuffer every X Puts.
/** /**
* Creates an object to access a HBase table. * Creates an object to access a HBase table.
@ -706,10 +707,20 @@ public class HTable implements HTableInterface, Closeable {
} }
private void doPut(final List<Put> puts) throws IOException { private void doPut(final List<Put> puts) throws IOException {
int n = 0;
for (Put put : puts) { for (Put put : puts) {
validatePut(put); validatePut(put);
writeBuffer.add(put); writeBuffer.add(put);
currentWriteBufferSize += put.heapSize(); currentWriteBufferSize += put.heapSize();
// we need to periodically see if the writebuffer is full instead of waiting until the end of the List
n++;
if (n == DOPUT_WB_CHECK) {
if (autoFlush || currentWriteBufferSize > writeBufferSize) {
flushCommits();
n = 0;
}
}
} }
if (autoFlush || currentWriteBufferSize > writeBufferSize) { if (autoFlush || currentWriteBufferSize > writeBufferSize) {
flushCommits(); flushCommits();

5
src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
View File

@ -190,8 +190,9 @@ public interface HTableInterface {
* until the internal buffer is full. * until the internal buffer is full.
* <p> * <p>
* This can be used for group commit, or for submitting user defined * This can be used for group commit, or for submitting user defined
* batches, but sending large lists of values is not recommended. That may * batches. The writeBuffer will be periodically inspected while the List
* produce performance problems. * is processed, so depending on the List size the writeBuffer may flush
* not at all, or more than once.
* @param puts The list of mutations to apply. The batch put is done by * @param puts The list of mutations to apply. The batch put is done by
* aggregating the iteration of the Puts over the write buffer * aggregating the iteration of the Puts over the write buffer
* at the client-side for a single RPC call. * at the client-side for a single RPC call.