8a5b3da292
HADOOP-2139 (phase 1) Increase parallelism in region servers
...
There are a lot of changes in this patch. The memcache has been changed from a per/region object to a per/column object, and HLocking has been removed since we do not have to maintain any locks across RPC calls.
This necessitated major changes to HRegion and HStore
Additionally there were many changes required to the unit tests since they tend to exploit some private interfaces that weren't designed to be public. Some of those interfaces changed so the test cases did as well.
This patch is the result of extensive analysis of the multiple threads in HBase that contend for shared resources: updates, reads, scanners, cache flushing, compaction and region splitting.
Many of the tests are timing sensitive, and since we tend to make "dormant" intervals as short as possible to speed up the Hudson build, we may go through several iterations of getting them right before Hudson is happy. This is especially true since two test cases failed on my dual cpu windows machine while running the tests under Ant, but ran fine under Eclipse.
However, now that the tests are passing locally, I believe the changes are doing the right thing, but may require some parameter tweaks.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@596835 13f79535-47bb-0310-9956-ffa450edef68
2007-11-20 21:53:30 +00:00
025ac21ab1
HADOOP-2040 Hudson hangs AFTER test has finished
...
Second commit against this jira
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@595406 13f79535-47bb-0310-9956-ffa450edef68
2007-11-15 19:38:17 +00:00
bf4262e220
HADOOP-2040 Hudson hangs AFTER test has finished
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@594715 13f79535-47bb-0310-9956-ffa450edef68
2007-11-13 23:38:21 +00:00
33ab394d4a
HADOOP-2176 Htable.deleteAll documentation is ambiguous
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@593708 13f79535-47bb-0310-9956-ffa450edef68
2007-11-10 01:09:33 +00:00
26b3d9442d
HADOOP-2161 getRow() is orders of magnitudes slower than get(), even on
...
rows with one column
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@593665 13f79535-47bb-0310-9956-ffa450edef68
2007-11-09 21:57:24 +00:00
5c869d23e9
HADOOP-2074 Simple switch to enable DEBUG level-logging in hbase
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592603 13f79535-47bb-0310-9956-ffa450edef68
2007-11-07 03:48:13 +00:00
656cee4138
HADOOP-2156 BufferUnderflowException for un-named HTableDescriptors
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592551 13f79535-47bb-0310-9956-ffa450edef68
2007-11-06 20:48:27 +00:00
9be624fc93
HADOOP-2155 Method expecting HBaseConfiguration throw NPE when given Configuration
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592549 13f79535-47bb-0310-9956-ffa450edef68
2007-11-06 20:47:35 +00:00
cda8c597fc
HADOOP-2157 Make Scanners implement Iterable
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@592548 13f79535-47bb-0310-9956-ffa450edef68
2007-11-06 20:45:28 +00:00
334402af58
HADOOP-2109
...
- Fix another race condition in processing dead servers,
- Fix error online meta regions: was using region name and not startKey as key for map.put.
- Change TestRegionServerExit to always kill the region server for the META region. This makes the test more deterministic and getting META reassigned was problematic.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591880 13f79535-47bb-0310-9956-ffa450edef68
2007-11-05 05:06:35 +00:00
41c71508db
HADOOP-2109 Fix another race condition in processing dead servers, Fix error online meta regions: was using region name and not startKey as key for map.put
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591722 13f79535-47bb-0310-9956-ffa450edef68
2007-11-04 01:47:24 +00:00
ad743978ed
HADOOP-2109 Fix another race condition in processing dead servers
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591549 13f79535-47bb-0310-9956-ffa450edef68
2007-11-03 04:00:12 +00:00
d8f4792f8e
HADOOP-2137 hql.jsp : The character 0x19 is not valid
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591177 13f79535-47bb-0310-9956-ffa450edef68
2007-11-02 00:43:07 +00:00
2bd1e2cac2
HADOOP-2109 Fixed race condition in processing server lease timeout.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591162 13f79535-47bb-0310-9956-ffa450edef68
2007-11-01 22:49:08 +00:00
e92a4023d5
HADOOP-2083 TestTableIndex failed in patch build #970 and #956
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@591039 13f79535-47bb-0310-9956-ffa450edef68
2007-11-01 15:01:43 +00:00
0c8412cb79
HADOOP-2126 Use Bob Jenkins' hash for bloom filters
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@590875 13f79535-47bb-0310-9956-ffa450edef68
2007-11-01 01:48:46 +00:00
e91c531235
HADOOP-2124 Use of `hostname` does not work on Cygwin in some cases
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@590273 13f79535-47bb-0310-9956-ffa450edef68
2007-10-30 19:08:23 +00:00
00d4ab27f3
HADOOP-2088 Make hbase runnable in $HADOOP_HOME/build(/contrib/hbase)
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@588699 13f79535-47bb-0310-9956-ffa450edef68
2007-10-26 16:16:53 +00:00
1aeb23aeee
HADOOP-2084 Add a LocalHBaseCluster
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@588264 13f79535-47bb-0310-9956-ffa450edef68
2007-10-25 16:12:51 +00:00
fa1c77bd3e
HADOOP-2079 HADOOP-2056 Fix generated HLog, HRegion names
...
HLog.splitLog was generating incorrect file names, HRegion was generating file names that could be far too long especially for local file systems, HMaster had a race condition in which an old HLog would get split by two threads simultaneously.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@586680 13f79535-47bb-0310-9956-ffa450edef68
2007-10-20 08:06:52 +00:00
cc202c4b74
HADOOP-2056 A table with row keys containing colon fails to split regions
...
Many modifications required because region names (which contain start key) were used as file or directory names. Now the startKey is encoded in an order preserving Base64 dialect.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@586159 13f79535-47bb-0310-9956-ffa450edef68
2007-10-18 22:21:09 +00:00
0b92e8826a
HADOOP-2064 TestSplit assertion and NPE failures (Patch build #952 and #953 )
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585293 13f79535-47bb-0310-9956-ffa450edef68
2007-10-17 00:09:55 +00:00
04cac52745
HADOOP-1872 TestCompaction assertions fail on some hosts
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585276 13f79535-47bb-0310-9956-ffa450edef68
2007-10-16 22:27:30 +00:00
a5701cbbb0
HADOOP-2061 Add new Base64 dialects - updated wrong portion of CHANGES.txt - fixed.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585225 13f79535-47bb-0310-9956-ffa450edef68
2007-10-16 19:11:07 +00:00
ab813c3673
HADOOP-2061 Add new Base64 dialects
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@585219 13f79535-47bb-0310-9956-ffa450edef68
2007-10-16 18:56:28 +00:00
8a381624d9
HADOOP-2059 In tests, exceptions in min dfs shutdown should not fail test
...
(e.g. nightly #272 )
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584919 13f79535-47bb-0310-9956-ffa450edef68
2007-10-15 21:05:23 +00:00
f07a9fd89a
HADOOP-2401 Add convenience put method that takes writable
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584844 13f79535-47bb-0310-9956-ffa450edef68
2007-10-15 18:42:28 +00:00
920c6b2021
HADOOP-2029 TestLogRolling fails too often in patch and nightlies
...
3rd commit against this issue.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584286 13f79535-47bb-0310-9956-ffa450edef68
2007-10-12 21:58:12 +00:00
8faa2ea9c6
HADOOP-2038 TestCleanRegionServerExit failed in patch build #927
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@584212 13f79535-47bb-0310-9956-ffa450edef68
2007-10-12 17:46:37 +00:00
8b387823be
HADOOP-2029 TestLogRolling fails too often in patch and nightlies
...
This is second commit against this issue.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583963 13f79535-47bb-0310-9956-ffa450edef68
2007-10-11 21:49:40 +00:00
00f32814d1
HADOOP-2035 Add logo to webapps
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583946 13f79535-47bb-0310-9956-ffa450edef68
2007-10-11 20:33:53 +00:00
642dd4294e
HADOOP-2029 TestLogRolling fails too often in patch and nightlies
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583839 13f79535-47bb-0310-9956-ffa450edef68
2007-10-11 14:45:36 +00:00
c9f105acd5
HADOOP-2017 TestRegionServerAbort failure in patch build #903 and
...
nightly #266
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583309 13f79535-47bb-0310-9956-ffa450edef68
2007-10-09 21:48:11 +00:00
5233c28aaa
HADOOP-2011 Make hbase daemon scripts take args in same order as hadoop
...
daemon scripts
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@583037 13f79535-47bb-0310-9956-ffa450edef68
2007-10-09 03:25:00 +00:00
79b3b1c6b2
HADOOP-2004 [hbase] webapp hql formatting bugs
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582867 13f79535-47bb-0310-9956-ffa450edef68
2007-10-08 16:25:33 +00:00
eb22494fb4
HADOOP-1957 Web UI with report on cluster state and basic browsing of tables
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582443 13f79535-47bb-0310-9956-ffa450edef68
2007-10-06 03:49:43 +00:00
6c9ddf9f37
HADOOP-1957 Web UI with report on cluster state and basic browsing of tables
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582442 13f79535-47bb-0310-9956-ffa450edef68
2007-10-06 03:09:50 +00:00
6e4efdf697
HADOOP-1937 When the master times out a region server's lease, it is too aggressive in reclaiming the server's log.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582165 13f79535-47bb-0310-9956-ffa450edef68
2007-10-05 09:39:32 +00:00
1acbfbde1f
HADOOP-1996 TestHStoreFile fails on windows if run multiple times
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@582066 13f79535-47bb-0310-9956-ffa450edef68
2007-10-05 03:26:32 +00:00
af8131b86d
HADOOP-1990 Regression test instability affects nightly and patch builds
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@581995 13f79535-47bb-0310-9956-ffa450edef68
2007-10-04 19:58:56 +00:00
c56582d19b
HADOOP-1960 If a region server cannot talk to the master before its lease times out, it should shut itself down
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@581396 13f79535-47bb-0310-9956-ffa450edef68
2007-10-02 21:42:07 +00:00
2f9acbbd45
HADOOP-1975 HBase tests failing with java.lang.NumberFormatException
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@581047 13f79535-47bb-0310-9956-ffa450edef68
2007-10-01 19:45:02 +00:00
188f9e6f23
HADOOP-1820 regionserver creates hlogs without bound
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580811 13f79535-47bb-0310-9956-ffa450edef68
2007-09-30 23:46:04 +00:00
8a3bc9c23e
HADOOP-1966 Make HBase unit tests more reliable in the Hudson environment.
...
Set hbase.root in test/hbase-site.xml; when running a test, the default does not work consistantly.
When a HBase mini cluster is started on top of an existing mini dfs cluster, it should not shut down the mini dfs cluster when the mini HBase cluster is shut down.
TestDFSAbort catches exceptions, prints the stack trace and re-throws the exception, so you can see when the exception happened in the log.
Catch runtime exceptions that were escaping from FSUtils.isFileSystemAvailable, enabling more reliable detection of dfs failure. HRegionServer also now checks to see if it is still accepting client requests.
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580745 13f79535-47bb-0310-9956-ffa450edef68
2007-09-30 16:09:38 +00:00
354c848546
HADOOP-1789 output formatting
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580487 13f79535-47bb-0310-9956-ffa450edef68
2007-09-28 21:10:49 +00:00
1236b87f48
HADOOP-1913 Build a Lucene index on an HBase table
...
Files I failed to add/delete on original commit
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580399 13f79535-47bb-0310-9956-ffa450edef68
2007-09-28 16:12:24 +00:00
7acd7d074a
HADOOP-1928 Have master pass the regionserver the filesystem to use
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@580166 13f79535-47bb-0310-9956-ffa450edef68
2007-09-27 21:28:31 +00:00
ccf42acf70
HADOOP-1941 StopRowFilter throws NPE when passed null row
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@579410 13f79535-47bb-0310-9956-ffa450edef68
2007-09-25 23:04:08 +00:00
87f305b608
HADOOP-1943 LogRolling test fails: reverting changes for HADOOP-1820
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@579353 13f79535-47bb-0310-9956-ffa450edef68
2007-09-25 19:13:50 +00:00
2cf41fe535
HADOOP-1820 Regionserver creates hlogs without bound
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/hadoop/trunk/src/contrib/hbase@579005 13f79535-47bb-0310-9956-ffa450edef68
2007-09-24 22:52:16 +00:00
Jim Kellerman
Michael Stack