context
Author: Yongqiang He
Summary:
https://issues.apache.org/jira/browse/HBASE-5521
As part of working on HBASE-5313, we want to add a new columnar encoder/decoder.
It makes sense to move compression to be part of encoder/decoder:
1) a scanner for a columnar encoded block can do lazy decompression to a
specific part of a key value object
2) avoid an extra bytes copy from encoder to hblock-writer.
If there is no encoder specified for a writer, the HBlock.Writer will use a
default compression-context to do something very similar to today's code.
Test Plan: existing unit tests verified by mbautin and tedyu. And no new test
added here since this code is just a preparation for columnar encoder. Will add
testcase later in that diff.
Reviewers: dhruba, tedyu, sc, mbautin
Reviewed By: mbautin
Differential Revision: https://reviews.facebook.net/D2097
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1302602 13f79535-47bb-0310-9956-ffa450edef68
Summary: Adding Java lint engine to HBase's .arcconfig. The lint engine itself
is part of the arc-jira repository that lives at
https://github.com/facebook/arc-jira/. There are some changes to be made there
to prevent lint from reporting errors for unmodified lines.
Test Plan: arc lint
Reviewers: nspiegelberg, Kannan, Karthik, JIRA, Liyin
Reviewed By: Liyin
Differential Revision: https://reviews.facebook.net/D2289
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1301751 13f79535-47bb-0310-9956-ffa450edef68
Author: Zhiqiu Kong
Summary:
Added two separate metrics for both get() and next(). This is done by
refactoring on internal next() API. To be more specific, only Get.get()
and ResultScanner.next() passes the metric name ("getsize" and
"nextsize" repectively) to
HRegion::RegionScanner::next(List<KeyValue>, String)
This will eventually hit StoreScanner()::next((List<KeyValue>,
int, String) where the metrics are counted.
And their call paths are:
1) Get
HTable::get(final Get get)
=> HRegionServer::get(byte [] regionName, Get get)
=> HRegion::get(final Get get, final Integer lockid)
=> HRegion::get(final Get get) [pass METRIC_GETSIZE to the
callee]
=> HRegion::RegionScanner::next(List<KeyValue> outResults, String
metric)
=> HRegion::RegionScanner::next(List<KeyValue> outResults, int limit,
String metric)
=> HRegion::RegionScanner::nextInternal(int limit, String metric)
=> KeyValueHeap::next(List<KeyValue> result, int limit, String
metric)
=> StoreScanner::next(List<KeyValue> outResult, int limit, String
metric)
2) Next
HTable::ClientScanner::next()
=> ScannerCallable::call()
=> HRegionServer::next(long scannerId)
=> HRegionServer::next(final long scannerId, int nbRows) [pass
METRIC_NEXTSIZE to the callee]
=> HRegion::RegionScanner::next(List<KeyValue> outResults, String
metric)
=> HRegion::RegionScanner::next(List<KeyValue> outResults, int limit,
String metric)
=> HRegion::RegionScanner::nextInternal(int limit, String metric)
=> KeyValueHeap::next(List<KeyValue> result, int limit, String
metric)
=> StoreScanner::next(List<KeyValue> outResult, int limit, String
metric)
Test Plan:
1. Passed unit tests.
2. Created a testcase TestRegionServerMetrics::testGetNextSize to
guarantee:
* Get/Next contributes to getsize/nextsize metrics
* Both getsize/nextsize are per Column Family
* Flush/compaction won't affect these two metrics
Reviewed By: mbautin
Reviewers: Kannan, mbautin, Liyin, JIRA
CC: Kannan, mbautin, Liyin, zhiqiu
Differential Revision: https://reviews.facebook.net/D1617
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1299147 13f79535-47bb-0310-9956-ffa450edef68
Author: Zhiqiu Kong
Summary:
The original 89-fb diff is: https://reviews.facebook.net/D1263
Slow opertaion log does not provide enough information when a filter is
present. The followings are done to add the filter info:
1) Added toString() method for filters inheriting FilterBase, this
affect 22 filters and their subclasses. The info added includes the
filter's name and its members. For example, for TimestampsFilter, we'll
output its class name as well as the defined timestamps.
2) Added a field 'filter' in Get::toMap() and Scan::toMap() to enable
the logging of filter info.
Task ID: #750975
Blame Rev:
Test Plan:
1. Run and passed unit-tests to make sure it does not break things
2. Run kannan's script to trigger the slow operation logging, checked
for each filter to make sure the filter info was logged. To be more
detailed, the output log are as following (only 'filter' filed is put
here for ease of reading):
"filter":"TimestampsFilter (3/3): [2, 3, 5]"
"filter":"TimestampsFilter (5/6): [2, 3, 5, 7, 11]"
"filter":"ColumnPrefixFilter col2"
"filter":"ColumnRangeFilter [col2a, col2b]"
"filter":"ColumnCountGetFilter 8"
"filter":"ColumnPaginationFilter (4, 4)"
"filter":"InclusiveStopFilter row"
"filter":"PrefixFilter row"
"filter":"PageFilter 1"
"filter":"SkipFilter TimestampsFilter (1/1): [1000]"
"filter":"WhileMatchFilter TimestampsFilter (3/3): [2, 3, 5]"
"filter":"KeyOnlyFilter"
"filter":"FirstKeyOnlyFilter"
"filter":"MultipleColumnPrefixFilter (3/3): [a, b, c]"
"filter":"DependentColumnFilter (family, qualifier, true, LESS, value)"
"filter":"FamilyFilter (LESS, value)"
"filter":"QualifierFilter (LESS, value)"
"filter":"RowFilter (LESS, value)"
"filter":"ValueFilter (LESS, value)"
"filter":"KeyOnlyFilter"
"filter":"FirstKeyOnlyFilter"
"filter":"SingleColumnValueFilter (family, qualifier, EQUAL, value)"
"filter":"SingleColumnValueExcludeFilter (family, qualifier, EQUAL,
value)"
"filter":"FilterList AND (2/2): [KeyOnlyFilter, FirstKeyOnlyFilter]"
Please check ~zhiqiu/Codes/scripts/testFilter.rb for the testing script.
3. Added unit test cases to TestOperation to verify the filters'
toString() method works well.
Reviewed By: mbautin
Reviewers: Kannan, madhuvaidya, mbautin, JIRA
CC: Kannan, madhuvaidya, mbautin, zhiqiu, stack
Differential Revision: https://reviews.facebook.net/D1539
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1299019 13f79535-47bb-0310-9956-ffa450edef68
Michael Stack
Zhihong Yu
larsh
Jean-Daniel Cryans
Jonathan Hsieh
mbautin
ramkrishna
Doug Meil