Commit Graph

14245 Commits

Author SHA1 Message Date
Michael Stack 73183e6b97 HADOOP-1531 Add RowFilter to HRegion.HScanner.
Adds a row/column filter interface and two implementations: A pager and a
row/column-value regex filter.
M src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java
M src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionInterface.java
    (openScanner): Add override that specifies a row fliter.
M src/contrib/hbase/src/java/org/apache/hadoop/hbase/HClient.java
    (obtainScanner): Add override that specifies a row fliter.
    (ColumnScanner): Add filter parameter to constructor.
M src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegion.java
    (getScanner): Add override with filter parameter.
    (next): Add handling of filtering.
A src/contrib/hbase/src/java/org/apache/hadoop/hbase/filter/InvalidRowFilterException.java
A src/contrib/hbase/src/java/org/apache/hadoop/hbase/filter/RegExpRowFilter.java
A src/contrib/hbase/src/java/org/apache/hadoop/hbase/filter/RowFilterSet.java
A src/contrib/hbase/src/java/org/apache/hadoop/hbase/filter/PageRowFilter.java
A src/contrib/hbase/src/java/org/apache/hadoop/hbase/filter/RowFilterInterface.java
    Row-filter interface, exception and implementations.
A src/contrib/hbase/src/test/org/apache/hadoop/hbase/filter/TestRegExpRowFilter.java
A src/contrib/hbase/src/test/org/apache/hadoop/hbase/filter/TestPageRowFilter.java
    Simple pager and regex filter tests.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@553620 13f79535-47bb-0310-9956-ffa450edef68
2007-07-05 19:50:04 +00:00
Jim Kellerman 9eb369c266 HADOOP-1560 NPE in MiniHBaseCluster on Windows
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@553080 13f79535-47bb-0310-9956-ffa450edef68
2007-07-04 04:48:18 +00:00
Michael Stack 655728f3bf HADOOP-1523 'Hung region servers waiting on write locks'
On shutdown, region servers and masters were just cancelling leases
without letting 'lease expired' code run -- code to clean up
outstanding locks in region server.  Outstanding read locks were
getting in the way of region server getting necessary write locks
needed for the shutdown process.  Also, cleaned up messaging around
shutdown so its clean -- no timeout messages as region servers try
to talk to a master that has already shutdown -- even when region
servers take their time going down.
M src/contrib/hbase/conf/hbase-default.xml
    Make region server timeout 30 seconds instead of 3 minutes.
    Clients retry anyways. Make so its likely region servers report
    in their shutdown message before their lease expires on master.
M src/contrib/hbase/src/java/org/apache/hadoop/hbase/Leases.java
    (closeAfterLeasesExpire): Added.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java
    Added comments.
    (stop): Converted from public to default access (master shuts
    down regionservers).
    (run): Use leases.closeAfterLeasesExpire instead of leases.close.
    Changed log of main thread exit from DEBUG to INFO.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMaster.java
    (letRegionsServersShutdown): Add better explaination of shutdown
    process to method doc.  Changed timeout waits from
    hbase.regionserver.msginterval to threadWakeFrequency.
    (regionServerReport): If closing, we used to immediately respond
    to region server with a MSG_REGIONSERVER_STOP.  This meant that
    we avoided handling of the region servers MSG_REPORT_EXITING sent
    on shutdown so region servers had no chance to cancel their lease
    in the master.  Reordered.  Moved sending of MSG_REGIONSERVER_STOP
    to after handling of MSG_REPORT_EXITING.  Also, in handling of
    MSG_REGIONSERER_STOP removed cancelling of leases.  Let leases
    expire normally (or get cancelled when the region server comes in
    with MSG_RPORT_EXITING).
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMsg.java
    (MSG_REGIONSERVER_STOP_IN_ARRAY): Added.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@552376 13f79535-47bb-0310-9956-ffa450edef68
2007-07-02 00:47:13 +00:00
Jim Kellerman 60a2e629d1 HADOOP-1519 map/reduce interface for HBase
AbstractMergeTestBase, HBaseTestCase: move createNewHRegion to HBaseTestCase
MiniHBaseCluster: add deleteOnExit, getDFSCluster, fix Javadoc
TestScanner2: moved KeyedData to org.apache.hadoop.hbase.io
TestTableMapReduce: new test case to test map/reduce interface to HBase
hbase-site.xml: change hbase.client.pause from 3 to 5 seconds, hbase.client.retries.number to 5 so that tests will not time out or run out of retries
HClient: moved KeyedData to org.apache.hadoop.hbase.io, fix javadoc, add method getStartKeys
HMaster: moved KeyedData to org.apache.hadoop.hbase.io, remove unused variables, remove extraneous throws clause, 
HRegionInterface, HRegionServer: moved KeyedData to org.apache.hadoop.hbase.io
KeyedData: moved KeyedData to org.apache.hadoop.hbase.io
KeyedDataArrayWritable: new class to support HBase map/reduce
org.apache.hadoop.hbase.mapred: new classes for map/reduce
- GroupingTableMap
- IdentityTableMap
- IdentityTableReduce
- TableInputFormat
- TableMap
- TableOutputCollector
- TableOutputFormat
- TableReduce
- TableSplit
hbase/bin/hbase: changes for map/reduce


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@552127 13f79535-47bb-0310-9956-ffa450edef68
2007-06-30 11:11:32 +00:00
Michael Stack 96c64712e6 HADOOP-1543 [base] Add HClient.tableExists
M    src/java/org/apache/hadoop/hbase/HClient.java
    (tableExists): Added
M    CHANGES.txt
M    src/java/org/apache/hadoop/hbase/package.html
    Fix javadoc.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@551725 13f79535-47bb-0310-9956-ffa450edef68
2007-06-28 22:29:07 +00:00
Jim Kellerman 2fa16d4fdf HADOOP-1537. Catch exceptions in testCleanRegionServerExit so we can see what is failing.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@551361 13f79535-47bb-0310-9956-ffa450edef68
2007-06-27 22:37:07 +00:00
Michael Stack a31d0e5f31 HADOOP-1534. [hbase] Memcache scanner fails if start key not present
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMemcache.java
    (findFirstRow): Change compare from startsWith -- looking for
    an 'exact' match -- to instead use compareTo and if >= 0, then
    we are at first key.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java
    Add logging of problematic directory if server won't start because
    log file exists.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@551039 13f79535-47bb-0310-9956-ffa450edef68
2007-06-27 04:50:10 +00:00
Jim Kellerman 6455364281 HADOOP-1509 Open HRegionServer/HClient for extension
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@550634 13f79535-47bb-0310-9956-ffa450edef68
2007-06-25 22:54:56 +00:00
Doug Cutting 31a8ba569b HADOOP-1511. Speed up hbase unit tests. Contributed by stack.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@549605 13f79535-47bb-0310-9956-ffa450edef68
2007-06-21 20:40:45 +00:00
Doug Cutting d1f85d739e HADOOP-1147. Remove @author tags from Java source files.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@549284 13f79535-47bb-0310-9956-ffa450edef68
2007-06-20 22:46:06 +00:00
Doug Cutting 2447ce1922 HADOOP-1498. Add file missed in prior commit.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@548794 13f79535-47bb-0310-9956-ffa450edef68
2007-06-19 16:40:59 +00:00
Doug Cutting eef130fc91 HADOOP-1498. Replace boxed types with primitives in many places. Contributed by stack.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@548523 13f79535-47bb-0310-9956-ffa450edef68
2007-06-18 22:59:14 +00:00
Jim Kellerman f503143412 Provide configurable per-column bloom filters - part 2.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@547468 13f79535-47bb-0310-9956-ffa450edef68
2007-06-15 00:39:34 +00:00
Jim Kellerman e78644ed10 HADOOP-1465 Add cluster stop/start scripts for hbase
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@547427 13f79535-47bb-0310-9956-ffa450edef68
2007-06-14 22:08:56 +00:00
Jim Kellerman 13c1e48253 HADOOP-1415 Integrate BSD licensed bloom filter implementation.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@547159 13f79535-47bb-0310-9956-ffa450edef68
2007-06-14 07:40:49 +00:00
Jim Kellerman cc8d45bd47 HADOOP-1469 Asychronous table creation
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@546635 13f79535-47bb-0310-9956-ffa450edef68
2007-06-12 21:08:27 +00:00
Jim Kellerman 0dd5f2759e HADOOP-1476 Distributed version of 'Performance Evaluation' script
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@546320 13f79535-47bb-0310-9956-ffa450edef68
2007-06-11 23:41:48 +00:00
Jim Kellerman 3f5229c66f HADOOP-1479 Fix NPE in HStore#get if store file only has keys < passed key.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@546275 13f79535-47bb-0310-9956-ffa450edef68
2007-06-11 20:52:29 +00:00
Jim Kellerman 09cf0a100f HADOOP-1421 HADOOP-1466 When a region server dies, its log file must be split up on a per region basis
so that region servers are assigned the regions have a log to apply edits from. Enhance fail over 
capabilities. 

For all the files modified, clean up javadoc, class method and field visibility.

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@546192 13f79535-47bb-0310-9956-ffa450edef68
2007-06-11 16:46:27 +00:00
Jim Kellerman 3509f88c48 HADOOP-1460 On shutdown IOException with complaint 'Cannot cancel lease that is not held'
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@544512 13f79535-47bb-0310-9956-ffa450edef68
2007-06-05 15:11:57 +00:00
Jim Kellerman ac718209e5 HADOOP-1445 Support updates across region splits and compactions
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@544188 13f79535-47bb-0310-9956-ffa450edef68
2007-06-04 17:14:10 +00:00
Jim Kellerman 7c3d11974b HADOOP-1391. Split test case for merge into two so it does not time out in Hudson
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@543862 13f79535-47bb-0310-9956-ffa450edef68
2007-06-03 08:52:06 +00:00
Jim Kellerman 66839c4c17 HADOOP-1391. Part 2 - table compaction via merging adjacent regions that have shrunk.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@543841 13f79535-47bb-0310-9956-ffa450edef68
2007-06-03 05:09:21 +00:00
Jim Kellerman a2fba1024d HADOOP-1391. Update CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@542595 13f79535-47bb-0310-9956-ffa450edef68
2007-05-29 17:23:15 +00:00
Jim Kellerman f35462842b HADOOP-1391. Part1: includes create/delete table; enable/disable table; add/remove column.
Patch has been tested locally and a new test has been added for administrative functions.

Still to do: merge regions.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@542592 13f79535-47bb-0310-9956-ffa450edef68
2007-05-29 17:17:44 +00:00
Jim Kellerman e4194e10e5 HADOOP-1430. HBase shutdown leaves regionservers up.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@541463 13f79535-47bb-0310-9956-ffa450edef68
2007-05-24 22:37:26 +00:00
Jim Kellerman 4f855ae28c HADOOP-1426. Make hbase scripts executable.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@541360 13f79535-47bb-0310-9956-ffa450edef68
2007-05-24 17:13:45 +00:00
Jim Kellerman af766ddc91 HADOOP-1426. Make hbase scripts executable + add test classes to CLASSPATH.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@541359 13f79535-47bb-0310-9956-ffa450edef68
2007-05-24 17:12:00 +00:00
Jim Kellerman 23f836454d HADOOP-1424. TestHBaseCluster fails with IllegalMonitorStateException. Fix regression introduced by HADOOP-1397.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@541095 13f79535-47bb-0310-9956-ffa450edef68
2007-05-23 21:30:25 +00:00
Jim Kellerman 19099c98b1 HADOOP-1420, HADOOP-1423. Findbugs changes, remove reference to removed class HLocking that broke main build.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@540973 13f79535-47bb-0310-9956-ffa450edef68
2007-05-23 15:14:00 +00:00
Jim Kellerman a658989730 HADOOP-1418. HBase miscellaneous: unit test for HClient, client to do 'Performance Evaluation', etc.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@540737 13f79535-47bb-0310-9956-ffa450edef68
2007-05-22 20:24:22 +00:00
Jim Kellerman e3e22fe37e HADOOP-1403. HBase reliability. Make master and region server more fault tolerant.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@540586 13f79535-47bb-0310-9956-ffa450edef68
2007-05-22 13:51:10 +00:00
Jim Kellerman 47bf7fd8bb HADOOP-1397. Replace custom hbase locking with java.util.concurrent.locks.ReentrantLock
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@540424 13f79535-47bb-0310-9956-ffa450edef68
2007-05-22 05:30:07 +00:00
Doug Cutting 356f3919e9 HADOOP-1386. Fix Path to not permit the empty string as a path, as this has lead to accidental file deletion. Instead force applications to use '.' to name the default directory. Contributed by Hairong.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@540359 13f79535-47bb-0310-9956-ffa450edef68
2007-05-21 23:30:40 +00:00
Jim Kellerman 8a53c236b9 HADOOP-1404. HBase command-line shutdown failing (Michael Stack)
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@540323 13f79535-47bb-0310-9956-ffa450edef68
2007-05-21 22:23:03 +00:00
Jim Kellerman ba1ef8c0c3 HADOOP-1402 Fix javadoc warnings in hbase contrib. (Michael Stack)
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@540271 13f79535-47bb-0310-9956-ffa450edef68
2007-05-21 20:35:19 +00:00
Jim Kellerman 7402e1b4fe HADOOP-1384 HBase change log. (Contrib committers cannot modify /CHANGES.txt)
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@539607 13f79535-47bb-0310-9956-ffa450edef68
2007-05-18 21:19:08 +00:00
Jim Kellerman a6b7be60d6 Hadoop-1384
HBase omnibus patch. Contributions by Vuk Ercegovac and Michael Stack.

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@539243 13f79535-47bb-0310-9956-ffa450edef68
2007-05-18 03:22:54 +00:00
Doug Cutting 3a70f2f97b HADOOP-1341. Fix intermittent failure of HBase unit test caused by deadlock. Contributed by Jim Kellerman.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@537292 13f79535-47bb-0310-9956-ffa450edef68
2007-05-11 21:11:43 +00:00
Doug Cutting add0c0c5a0 HADOOP-1325. First complete, functioning version of HBase. Contributed by Jim Kellerman.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@535970 13f79535-47bb-0310-9956-ffa450edef68
2007-05-07 19:58:53 +00:00
Doug Cutting ec9a568764 HADOOP-1282. Omnibus HBase patch. Improved tests and configuration. Contributed by Jim Kellerman.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@532083 13f79535-47bb-0310-9956-ffa450edef68
2007-04-24 21:13:08 +00:00
Doug Cutting 114d67c614 HADOOP-1148. More indentation and spacing fixes.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@530556 13f79535-47bb-0310-9956-ffa450edef68
2007-04-19 21:34:41 +00:00
Doug Cutting 4a8d243f4e HADOOP-1148. More re-indentation.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@529756 13f79535-47bb-0310-9956-ffa450edef68
2007-04-17 20:37:59 +00:00
Thomas White 89af8294f4 HADOOP-1203. Change UpgradeUtilities used by DFS tests to use MiniDFSCluster to start and stop NameNode/DataNodes. Contributed by Nigel Daley.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@526281 13f79535-47bb-0310-9956-ffa450edef68
2007-04-06 21:09:28 +00:00
Doug Cutting 948d0cdec4 HADOOP-1045. Add contrib/hbase, a BigTable-like online database.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@525267 13f79535-47bb-0310-9956-ffa450edef68
2007-04-03 20:34:28 +00:00