Commit Graph

18537 Commits

Author SHA1 Message Date
Jim Kellerman cbe167c981 HADOOP-2139 (phase 2) Make region server more event driven
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@597959 13f79535-47bb-0310-9956-ffa450edef68
2007-11-25 07:17:38 +00:00
Michael Stack 9052780e2d HADOOP-2267 Change the hbase shell prompt's title
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@597725 13f79535-47bb-0310-9956-ffa450edef68
2007-11-23 19:16:45 +00:00
Jim Kellerman 8a5b3da292 HADOOP-2139 (phase 1) Increase parallelism in region servers
There are a lot of changes in this patch. The memcache has been changed from a per/region object to a per/column object, and HLocking has been removed since we do not have to maintain any locks across RPC calls.

This necessitated major changes to HRegion and HStore

Additionally there were many changes required to the unit tests since they tend to exploit some private interfaces that weren't designed to be public. Some of those interfaces changed so the test cases did as well.

This patch is the result of extensive analysis of the multiple threads in HBase that contend for shared resources: updates, reads, scanners, cache flushing, compaction and region splitting.

Many of the tests are timing sensitive, and since we tend to make "dormant" intervals as short as possible to speed up the Hudson build, we may go through several iterations of getting them right before Hudson is happy. This is especially true since two test cases failed on my dual cpu windows machine while running the tests under Ant, but ran fine under Eclipse.

However, now that the tests are passing locally, I believe the changes are doing the right thing, but may require some parameter tweaks.

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@596835 13f79535-47bb-0310-9956-ffa450edef68
2007-11-20 21:53:30 +00:00
Michael Stack 025ac21ab1 HADOOP-2040 Hudson hangs AFTER test has finished
Second commit against this jira


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@595406 13f79535-47bb-0310-9956-ffa450edef68
2007-11-15 19:38:17 +00:00
Michael Stack bf4262e220 HADOOP-2040 Hudson hangs AFTER test has finished
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@594715 13f79535-47bb-0310-9956-ffa450edef68
2007-11-13 23:38:21 +00:00
Michael Stack 33ab394d4a HADOOP-2176 Htable.deleteAll documentation is ambiguous
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@593708 13f79535-47bb-0310-9956-ffa450edef68
2007-11-10 01:09:33 +00:00
Michael Stack 26b3d9442d HADOOP-2161 getRow() is orders of magnitudes slower than get(), even on
rows with one column


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@593665 13f79535-47bb-0310-9956-ffa450edef68
2007-11-09 21:57:24 +00:00
Michael Stack 5c869d23e9 HADOOP-2074 Simple switch to enable DEBUG level-logging in hbase
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592603 13f79535-47bb-0310-9956-ffa450edef68
2007-11-07 03:48:13 +00:00
Michael Stack 656cee4138 HADOOP-2156 BufferUnderflowException for un-named HTableDescriptors
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592551 13f79535-47bb-0310-9956-ffa450edef68
2007-11-06 20:48:27 +00:00
Michael Stack 9be624fc93 HADOOP-2155 Method expecting HBaseConfiguration throw NPE when given Configuration
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592549 13f79535-47bb-0310-9956-ffa450edef68
2007-11-06 20:47:35 +00:00
Michael Stack cda8c597fc HADOOP-2157 Make Scanners implement Iterable
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592548 13f79535-47bb-0310-9956-ffa450edef68
2007-11-06 20:45:28 +00:00
Jim Kellerman 334402af58 HADOOP-2109
- Fix another race condition in processing dead servers,
- Fix error online meta regions: was using region name and not startKey as key for map.put.
- Change TestRegionServerExit to always kill the region server for the META region. This makes the test more deterministic and getting META reassigned was problematic.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591880 13f79535-47bb-0310-9956-ffa450edef68
2007-11-05 05:06:35 +00:00
Jim Kellerman 41c71508db HADOOP-2109 Fix another race condition in processing dead servers, Fix error online meta regions: was using region name and not startKey as key for map.put
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591722 13f79535-47bb-0310-9956-ffa450edef68
2007-11-04 01:47:24 +00:00
Jim Kellerman ad743978ed HADOOP-2109 Fix another race condition in processing dead servers
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591549 13f79535-47bb-0310-9956-ffa450edef68
2007-11-03 04:00:12 +00:00
Michael Stack d8f4792f8e HADOOP-2137 hql.jsp : The character 0x19 is not valid
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591177 13f79535-47bb-0310-9956-ffa450edef68
2007-11-02 00:43:07 +00:00
Jim Kellerman 2bd1e2cac2 HADOOP-2109 Fixed race condition in processing server lease timeout.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591162 13f79535-47bb-0310-9956-ffa450edef68
2007-11-01 22:49:08 +00:00
Michael Stack e92a4023d5 HADOOP-2083 TestTableIndex failed in patch build #970 and #956
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591039 13f79535-47bb-0310-9956-ffa450edef68
2007-11-01 15:01:43 +00:00
Jim Kellerman 0c8412cb79 HADOOP-2126 Use Bob Jenkins' hash for bloom filters
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@590875 13f79535-47bb-0310-9956-ffa450edef68
2007-11-01 01:48:46 +00:00
Jim Kellerman e91c531235 HADOOP-2124 Use of `hostname` does not work on Cygwin in some cases
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@590273 13f79535-47bb-0310-9956-ffa450edef68
2007-10-30 19:08:23 +00:00
Michael Stack 00d4ab27f3 HADOOP-2088 Make hbase runnable in $HADOOP_HOME/build(/contrib/hbase)
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@588699 13f79535-47bb-0310-9956-ffa450edef68
2007-10-26 16:16:53 +00:00
Michael Stack 1aeb23aeee HADOOP-2084 Add a LocalHBaseCluster
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@588264 13f79535-47bb-0310-9956-ffa450edef68
2007-10-25 16:12:51 +00:00
Jim Kellerman fa1c77bd3e HADOOP-2079 HADOOP-2056 Fix generated HLog, HRegion names
HLog.splitLog was generating incorrect file names, HRegion was generating file names that could be far too long especially for local file systems, HMaster had a race condition in which an old HLog would get split by two threads simultaneously. 

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@586680 13f79535-47bb-0310-9956-ffa450edef68
2007-10-20 08:06:52 +00:00
Jim Kellerman cc202c4b74 HADOOP-2056 A table with row keys containing colon fails to split regions
Many modifications required because region names (which contain start key) were used as file or directory names. Now the startKey is encoded in an order preserving Base64 dialect.

git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@586159 13f79535-47bb-0310-9956-ffa450edef68
2007-10-18 22:21:09 +00:00
Michael Stack 0b92e8826a HADOOP-2064 TestSplit assertion and NPE failures (Patch build #952 and #953)
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585293 13f79535-47bb-0310-9956-ffa450edef68
2007-10-17 00:09:55 +00:00
Michael Stack 04cac52745 HADOOP-1872 TestCompaction assertions fail on some hosts
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585276 13f79535-47bb-0310-9956-ffa450edef68
2007-10-16 22:27:30 +00:00
Jim Kellerman a5701cbbb0 HADOOP-2061 Add new Base64 dialects - updated wrong portion of CHANGES.txt - fixed.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585225 13f79535-47bb-0310-9956-ffa450edef68
2007-10-16 19:11:07 +00:00
Jim Kellerman ab813c3673 HADOOP-2061 Add new Base64 dialects
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585219 13f79535-47bb-0310-9956-ffa450edef68
2007-10-16 18:56:28 +00:00
Michael Stack 8a381624d9 HADOOP-2059 In tests, exceptions in min dfs shutdown should not fail test
(e.g. nightly #272)


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584919 13f79535-47bb-0310-9956-ffa450edef68
2007-10-15 21:05:23 +00:00
Michael Stack f07a9fd89a HADOOP-2401 Add convenience put method that takes writable
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584844 13f79535-47bb-0310-9956-ffa450edef68
2007-10-15 18:42:28 +00:00
Michael Stack 920c6b2021 HADOOP-2029 TestLogRolling fails too often in patch and nightlies
3rd commit against this issue.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584286 13f79535-47bb-0310-9956-ffa450edef68
2007-10-12 21:58:12 +00:00
Michael Stack 8faa2ea9c6 HADOOP-2038 TestCleanRegionServerExit failed in patch build #927
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584212 13f79535-47bb-0310-9956-ffa450edef68
2007-10-12 17:46:37 +00:00
Michael Stack 8b387823be HADOOP-2029 TestLogRolling fails too often in patch and nightlies
This is second commit against this issue.


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583963 13f79535-47bb-0310-9956-ffa450edef68
2007-10-11 21:49:40 +00:00
Michael Stack 00f32814d1 HADOOP-2035 Add logo to webapps
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583946 13f79535-47bb-0310-9956-ffa450edef68
2007-10-11 20:33:53 +00:00
Michael Stack 642dd4294e HADOOP-2029 TestLogRolling fails too often in patch and nightlies
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583839 13f79535-47bb-0310-9956-ffa450edef68
2007-10-11 14:45:36 +00:00
Michael Stack c9f105acd5 HADOOP-2017 TestRegionServerAbort failure in patch build #903 and
nightly #266


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583309 13f79535-47bb-0310-9956-ffa450edef68
2007-10-09 21:48:11 +00:00
Michael Stack 5233c28aaa HADOOP-2011 Make hbase daemon scripts take args in same order as hadoop
daemon scripts


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583037 13f79535-47bb-0310-9956-ffa450edef68
2007-10-09 03:25:00 +00:00
Michael Stack 79b3b1c6b2 HADOOP-2004 [hbase] webapp hql formatting bugs
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582867 13f79535-47bb-0310-9956-ffa450edef68
2007-10-08 16:25:33 +00:00
Michael Stack eb22494fb4 HADOOP-1957 Web UI with report on cluster state and basic browsing of tables
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582443 13f79535-47bb-0310-9956-ffa450edef68
2007-10-06 03:49:43 +00:00
Michael Stack 6c9ddf9f37 HADOOP-1957 Web UI with report on cluster state and basic browsing of tables
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582442 13f79535-47bb-0310-9956-ffa450edef68
2007-10-06 03:09:50 +00:00
Jim Kellerman 6e4efdf697 HADOOP-1937 When the master times out a region server's lease, it is too aggressive in reclaiming the server's log.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582165 13f79535-47bb-0310-9956-ffa450edef68
2007-10-05 09:39:32 +00:00
Jim Kellerman 1acbfbde1f HADOOP-1996 TestHStoreFile fails on windows if run multiple times
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582066 13f79535-47bb-0310-9956-ffa450edef68
2007-10-05 03:26:32 +00:00
Jim Kellerman af8131b86d HADOOP-1990 Regression test instability affects nightly and patch builds
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@581995 13f79535-47bb-0310-9956-ffa450edef68
2007-10-04 19:58:56 +00:00
Jim Kellerman c56582d19b HADOOP-1960 If a region server cannot talk to the master before its lease times out, it should shut itself down
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@581396 13f79535-47bb-0310-9956-ffa450edef68
2007-10-02 21:42:07 +00:00
Jim Kellerman 2f9acbbd45 HADOOP-1975 HBase tests failing with java.lang.NumberFormatException
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@581047 13f79535-47bb-0310-9956-ffa450edef68
2007-10-01 19:45:02 +00:00
Jim Kellerman 188f9e6f23 HADOOP-1820 regionserver creates hlogs without bound
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580811 13f79535-47bb-0310-9956-ffa450edef68
2007-09-30 23:46:04 +00:00
Jim Kellerman 8a3bc9c23e HADOOP-1966 Make HBase unit tests more reliable in the Hudson environment.
Set hbase.root in test/hbase-site.xml; when running a test, the default does not work consistantly.

When a HBase mini cluster is started on top of an existing mini dfs cluster, it should not shut down the mini dfs cluster when the mini HBase cluster is shut down.

TestDFSAbort catches exceptions, prints the stack trace and re-throws the exception, so you can see when the exception happened in the log.

Catch runtime exceptions that were escaping from FSUtils.isFileSystemAvailable, enabling more reliable detection of dfs failure. HRegionServer also now checks to see if it is still accepting client requests.



git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580745 13f79535-47bb-0310-9956-ffa450edef68
2007-09-30 16:09:38 +00:00
Michael Stack 354c848546 HADOOP-1789 output formatting
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580487 13f79535-47bb-0310-9956-ffa450edef68
2007-09-28 21:10:49 +00:00
Michael Stack 1236b87f48 HADOOP-1913 Build a Lucene index on an HBase table
Files I failed to add/delete on original commit


git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580399 13f79535-47bb-0310-9956-ffa450edef68
2007-09-28 16:12:24 +00:00
Michael Stack 7acd7d074a HADOOP-1928 Have master pass the regionserver the filesystem to use
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580166 13f79535-47bb-0310-9956-ffa450edef68
2007-09-27 21:28:31 +00:00
Michael Stack ccf42acf70 HADOOP-1941 StopRowFilter throws NPE when passed null row
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@579410 13f79535-47bb-0310-9956-ffa450edef68
2007-09-25 23:04:08 +00:00