Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-1576 TIF needs to be able to set scanner caching size for smaller row tables & performance 

git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@787917 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Michael Stack 2009-06-24 06:02:39 +00:00
parent 44d6294ed7
commit cc920fe6bb
3 changed files with 27 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGES.txt
View File

@ -393,6 +393,8 @@ Release 0.20.0 - Unreleased
HBASE-1532 UI Visibility into ZooKeeper
HBASE-1572 Zookeeper log4j property set to ERROR on default, same output
when cluster working and not working (Jon Gray via Stack)
HBASE-1576 TIF needs to be able to set scanner caching size for smaller
row tables & performance
OPTIMIZATIONS
HBASE-1412 Change values for delete column and column family in KeyValue

14
src/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java
View File

@ -168,5 +168,17 @@ public class TableMapReduceUtil {
int regions = outputTable.getRegionsInfo().size();
job.setNumMapTasks(regions);
}
/**
* Sets the number of rows to return and cache with each scanner iteration.
* Higher caching values will enable faster mapreduce jobs at the expense of
* requiring more heap to contain the cached rows.
*
* @param job The current job configuration to adjust.
* @param batchSize The number of rows to return in batch with each scanner
* iteration.
*/
public static void setScannerCaching(JobConf job, int batchSize) {
job.setInt("hbase.client.scanner.caching", batchSize);
}
}

12
src/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
View File

@ -168,4 +168,16 @@ public class TableMapReduceUtil {
job.setNumMapTasks(regions);
}
/**
* Sets the number of rows to return and cache with each scanner iteration.
* Higher caching values will enable faster mapreduce jobs at the expense of
* requiring more heap to contain the cached rows.
*
* @param job The current job configuration to adjust.
* @param batchSize The number of rows to return in batch with each scanner
* iteration.
*/
public static void setScannerCaching(JobConf job, int batchSize) {
job.setInt("hbase.client.scanner.caching", batchSize);
}
}