Commit Graph

18263 Commits

Author SHA1 Message Date
Michael Stack 8fc5cd2b95 HBASE-558 Output hbase+hadoop+jvm version as well as java opts, ulimit, into master/regionserver log on startup
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656798 13f79535-47bb-0310-9956-ffa450edef68
2008-05-15 19:22:54 +00:00
Michael Stack b187c39355 HBASE-593 build on hudson failing compiling jsp -- make it so we do jspc once per invocation
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656794 13f79535-47bb-0310-9956-ffa450edef68
2008-05-15 18:53:12 +00:00
Michael Stack 1f3606e5b0 HBASE-593 build on hudson failing compiling jsp
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656518 13f79535-47bb-0310-9956-ffa450edef68
2008-05-15 06:48:37 +00:00
Michael Stack bcdb066cae Another nothing change to provoke a build
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656514 13f79535-47bb-0310-9956-ffa450edef68
2008-05-15 05:43:28 +00:00
Michael Stack 7ddd733236 HBASE-569 DemoClient.php
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656511 13f79535-47bb-0310-9956-ffa450edef68
2008-05-15 05:26:47 +00:00
Michael Stack 100a1616d4 Nothing change to force build on hudson to see if ant changes will make it build again
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656506 13f79535-47bb-0310-9956-ffa450edef68
2008-05-15 05:02:02 +00:00
Michael Stack 302997693a Add note about the 0.1.2 release
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656500 13f79535-47bb-0310-9956-ffa450edef68
2008-05-15 03:56:43 +00:00
Michael Stack 99fc06705d HBASE-23 Fix bug introduced by my edits of JDC's patch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656456 13f79535-47bb-0310-9956-ffa450edef68
2008-05-14 22:41:58 +00:00
Michael Stack 822ba8bc3b HBASE-23 UI listing regions should be sorted by address and show additional region state
M  src/java/org/apache/hadoop/hbase/HServerLoad.java
    (toString): Overload that takes interval.
M  src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
    (getConfiguration): Added
M  src/java/org/apache/hadoop/hbase/master/HMaster.java
    (getAverageLoad): Added
M src/java/org/apache/hadoop/hbase/HServerAddress.java
    (getHostname): Added.
M  src/java/org/apache/hadoop/hbase/client/HTable.java
    (getRegionsInfo): Added.
M  src/webapps/regionserver/regionserver.jsp
M  src/webapps/master/master.jsp
    Output requests as requests per second.
    Fixed up the help comments.  Changed all to use new getHostname
    rather than IPs.  Added encoded name column.
M  src/webapps/master/table.jsp
    Added.
M  src/webapps/master/WEB-INF/web.xml
    Add new tables servlet.
M  bin/hbase
    Put hbase in front of hadoop again.  FIxes not being able to find
    stylesheets -- might break logging again though doesnt seem to in
    tests.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656366 13f79535-47bb-0310-9956-ffa450edef68
2008-05-14 19:06:52 +00:00
Jim Kellerman 595b75b105 HBASE-538 Improve exceptions that come out on client-side
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@656341 13f79535-47bb-0310-9956-ffa450edef68
2008-05-14 18:07:03 +00:00
Jim Kellerman 8d5e1e7025 HBASE-624 Master will shut down if number of active region servers is zero even if shutdown was not requested
M HMaster

- Moved HMaster.quiescedMetaServers to ServerManager.quiescedServers and changed name since only servers serving user regions get quiesced.
- Removed HMaster.tableInCreation - not used

M ServerManager

- Don't check if quiescedServers.get() >= serversToServerInfo.size() unless master.shutdownRequested is true.

M HRegionServer

- Change order of checks in main loop of HRegionServer.run, so that booleans are checked before we check the number of messages to process
- Don't break out of main loop if restart or stop requested - stop is checked at top of loop and we need to continue rather than break out of main loop if restart was requested



git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@655962 13f79535-47bb-0310-9956-ffa450edef68
2008-05-13 18:25:31 +00:00
Jim Kellerman 197cf5c7d7 HBASE-622 Remove StaticTestEnvironment and put a log4j.properties in src/test
Set logging level for Hadoop to WARN

git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@655907 13f79535-47bb-0310-9956-ffa450edef68
2008-05-13 15:18:59 +00:00
Jim Kellerman 4e03053213 HBASE-622 Remove StaticTestEnvironment and put a log4j.properties in src/test
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@655650 13f79535-47bb-0310-9956-ffa450edef68
2008-05-12 21:22:38 +00:00
Jim Kellerman 8544e3d1f3 HBASE-614 Retiring regions is not used; exploit or remove
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654653 13f79535-47bb-0310-9956-ffa450edef68
2008-05-08 23:41:47 +00:00
Michael Stack 7eddf55102 HBASE-620 testmergetool failing in branch and trunk since hbase-618 went in
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654327 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 23:33:33 +00:00
Michael Stack a390d36ef5 HBASE-620 testmergetool failing in branch and trunk since hbase-618 went in
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654326 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 23:33:18 +00:00
Jim Kellerman 5a8da86833 HBASE-611 regionserver should do basic health check before reporting alls-well to the master
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654301 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 22:08:21 +00:00
Jim Kellerman 64842079e0 HBASE-453 undeclared throwable exception from HTable.get
Instead of wrapping UndeclaredThrowableException in a new IOException, wrap UndeclaredThrowableException.getCause

git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654267 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 20:39:14 +00:00
Jim Kellerman 4aa78f4f28 HBASE-453 undeclared throwable exception from HTable.get
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654256 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 20:19:53 +00:00
Jim Kellerman eaa03f245b HBASE-600 Filters have excessive DEBUG logging
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654229 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 19:35:47 +00:00
Jim Kellerman b3f4ddc1bc HBASE-478 offlining of table does not run reliably
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654193 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 17:56:06 +00:00
Michael Stack cfefc60bac HBASE-619 Fix 'logs' link in UI
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654184 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 17:31:29 +00:00
Michael Stack 20ee89aa4c HBASE-618 We always compact if 2 files, regardless of the compaction threshold setting
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@654169 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 16:58:10 +00:00
Michael Stack 2eef957d36 HBASE-47 option to set TTL for columns in hbase
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@653986 13f79535-47bb-0310-9956-ffa450edef68
2008-05-07 04:17:09 +00:00
Jim Kellerman 8df7f79781 HBASE-405 TIF and TOF use log4j directly rather than apache commons-logging
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@653941 13f79535-47bb-0310-9956-ffa450edef68
2008-05-06 22:18:24 +00:00
Jim Kellerman ab778e22e4 HBASE-607 MultiRegionTable.makeMultiRegionTable is not deterministic enough for regression tests
M MultiRegionTable

  Make deterministic by creating the regions directly and not rely on
  the asychronous nature of cache flushes, compactions and splits. The
  regions are small, but the point of this class is to generate a
  table with multiple regions so we can test map / reduce, region
  onlining / offlining, etc.

  Removed PUNCTUATION from row keys. Not sure why it was there in the
  first place, other than perhaps to verify that a row key can have
  punctuation in it provided it is not the first character. This will
  become moot when row keys change from Text to byte[] anyways.

  Incorporate repeated code
{code}
    region.close();
    region.getLog().closeAndDelete();
{code}
  into private method closeRegionAndDeleteLog

M TestSplit

  extends HBaseClusterTestCase instead of MultiRegionTable. It didn't
  use the output of MultiRegionTable, so all that work was just wasted
  by this test.

M TestTableIndex, TestTableMapReduce

  The only two tests that currently use MultiRegionTable. Minor
  modifications needed because MultiRegionTable now handles starting
  and stopping of the mini-DFS cluster. With the new MultiRegionTable
  class, if these tests fail now it will be because something they are
  testing has regressed and not because MultiRegionTable failed.



git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@652587 13f79535-47bb-0310-9956-ffa450edef68
2008-05-01 17:37:24 +00:00
Michael Stack c70e4fa984 HBASE-609 Master doesn't see regionserver edits because of clock skew
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@652436 13f79535-47bb-0310-9956-ffa450edef68
2008-05-01 06:05:58 +00:00
Michael Stack 0c08836299 HBASE-608 HRegionServer::getThisIP() checks hadoop config var for dns interface name
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@652428 13f79535-47bb-0310-9956-ffa450edef68
2008-05-01 04:22:56 +00:00
Michael Stack 9cd46728ff HBASE-604 Don't allow CLASSPATH from environment pollute the hbase CLASSPATH
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@652189 13f79535-47bb-0310-9956-ffa450edef68
2008-04-29 23:29:17 +00:00
Bryan Duxbury 1deedc318a HBASE-603 When an exception bubbles out of getRegionServerWithRetries, wrap the exception with a RetriesExhaustedException
-Added RetriesExhaustedException to client package
-HTable now throws REE when retries run out

git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@652185 13f79535-47bb-0310-9956-ffa450edef68
2008-04-29 23:03:20 +00:00
Michael Stack 21da84cfea HBASE-601 Just remove deprecated methods in HTable; 0.2 is not backward compatible anyways
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@652054 13f79535-47bb-0310-9956-ffa450edef68
2008-04-28 04:58:09 +00:00
Michael Stack 495467870b Remove useless, annoying DEBUG (previously added by me)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@651740 13f79535-47bb-0310-9956-ffa450edef68
2008-04-25 23:43:49 +00:00
Michael Stack cecc9e99ff HBASE-581 Allow adding filters to TableInputFormat (At same time, ensure TIF is subclassable)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@651368 13f79535-47bb-0310-9956-ffa450edef68
2008-04-24 19:48:32 +00:00
Michael Stack fb2db8cdaa HBASE-588 Still a 'hole' in scanners, even after HBASE-532
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@651018 13f79535-47bb-0310-9956-ffa450edef68
2008-04-23 19:22:15 +00:00
Michael Stack e1da900df8 HBASE-588 Still a 'hole' in scanners, even after HBASE-532
Add a ChangedReadersObserver interface.  HStore notifies registered
observers when list of HStoreFile Readers changes -- at flush time
and at compaction time.  Scanners are only current observers.

Also fix a deadlock flushing by changing lock types and moving flush
request out from under lock.

M  src/test/org/apache/hadoop/hbase/MultiRegionTable.java
    (getCacheFlushListern): Renamed as getFlushRequester
M  src/java/org/apache/hadoop/hbase/regionserver/HStore.java
    Added synchronizations on this.storefiles.
    (activeScanners, newScannerLock): Removed.  We no longer try to
    block out scanners when compacting (Turns out same functionality
    for blocking scanners is up in HRegion used around closing/split).
    (changedReadersObservers): Added.
    (updateReaders): New method that manages the insertion of new
    reader on flush.  Also calls new notifyChangedReadersObservers.
    (notifyChagnedReadersObservers, addChangedReadersObserver,
      deleteChangedReadersObservers): Added.
    (completeCompaction): Previous deleting old store files, we'd
    remove from this.storefiles and delete all in one step; now we
    do the remove first, notify all observers of readers, and then
    do the delete so observers have a chance to clean up any old
    references to files about to be deleted.  Removed all the lockout
    of new scanner creation and wait on old scanners to come int.
    (updateActiveScanners): Removed.
    (getStorefiles): Accessor.  Added.
M  src/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
    Added implementation of new ChangedReadersObserver interface.
    Added a lock that we hold when 'nexting' and when changing the
    set of readers out from under the scanner.
    Changed the constructor moving bulk into new openReaders method
    that we reuse when list of Readers changes.
    (next): Moved head of this method into new getNextViableRow (used
    to be called chosenRow and chosenTimestamp).  New method returns
    simple datastructure of row and timestamp (ViableRow).
    (close): On close, remove ourselves as ChangedReadersObserver (we
    added ourselves in the constructor).
    (updateReaders): Changed the set of Readers out from under the
    Scanner.
A  src/java/org/apache/hadoop/hbase/regionserver/ChangedReadersObserver.java
    Added.
M  src/java/org/apache/hadoop/hbase/regionserver/Flusher.java
    Changed name of the interface we implement from CacheFlushListener to
    FlushRequester.
D  src/java/org/apache/hadoop/hbase/regionserver/CacheFlushListener.java
    Renamed as FlushRequester.
M  src/java/org/apache/hadoop/hbase/regionserver/HStoreScanner.java
    Remove update of activeScanners.
A  src/java/org/apache/hadoop/hbase/regionserver/FlushRequester.java
    Added.  Rename of CacheFlushListener.
M  src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
    Renamed method getCacheFlushListener as getFlushRequester.
M src/java/org/apache/hadoop/hbase/regionserver/HAbstractScanner.java
    Formatting.
M  src/java/org/apache/hadoop/hbase/regionserver/HRegion.java
    CacheFlushListener was renamed as FlushListener.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@651017 13f79535-47bb-0310-9956-ffa450edef68
2008-04-23 19:21:26 +00:00
Jim Kellerman 3607d3e0db HBASE-584 Names in the filter interface are confusing (Clint Morgan via Jim Kellerman) (API change for filters)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@650711 13f79535-47bb-0310-9956-ffa450edef68
2008-04-23 01:03:38 +00:00
Michael Stack 2bf301a4df HBASE-596 DemoClient.py
Added licenses to examples along w/ committing new Python example.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@650333 13f79535-47bb-0310-9956-ffa450edef68
2008-04-22 00:39:02 +00:00
Michael Stack cffa3a254e HBASE-586 HRegion runs HStore memcache snapshotting -- fix it so only HStore
knows about workings of memcache

HStore knows about workings of memcache

This patch moves the running of mecache snapshots out of the control
of HRegion and hides the memcache details in HStore.   This patch also
does a bunch of work on the merge tool fixing a bug in the metautils along
the way.  The merge tool test was failing.  We weren't setting into the
HLog the maximum sequence id after we'd opened a region -- as HRS does --
so were losing edits. On the way, refactored the merge tool test to get
rid of duplicated code.  Finally, cleans up logging in HStore to aid
debugging; e.g. we always refer to the sequence id as the 'sequence id'
in log messages rather than as seqId or 'sequence record' so can sort
log as see state of sequence id transitions.

Version 2 changes the order in which things are run in memcache.  532
made it so flushing did snapshot and then cleared the snapshot.  Now,
we snapshot before we flush a store, then inside in the store flush,
we call getSnapshot and then clearSnapshot.

M  src/test/org/apache/hadoop/hbase/regionserver/TestHMemcache.java
  How snapshotting changed.  Change test in accordance.
M  src/test/org/apache/hadoop/hbase/regionserver/TestHLog.java
  Use accessor to get file number.
M  src/test/org/apache/hadoop/hbase/util/TestMergeTool.java
  Refactored to remove duplicated code so could tell what was going on.
  (mergeAndVerify, verifyMerge): Addd.
M  src/java/org/apache/hadoop/hbase/regionserver/Memcache.java
  (snapshot): Changed so it no longer returns snapshot.
M  src/java/org/apache/hadoop/hbase/regionserver/HStore.java
  Changed log messages removing the useless and adding info to others.
  (this.maxSeqId): We used to add 1 to this in here in HStore.  Let
  HRegion do it.  Its the one that does the machinations w/ sequenceids
  anyways.  Make flushes return the amount flushed.  Use this updating
  the regions memcacheSize accounting.
  (snapshot): Added method for the region to call.
M  src/java/org/apache/hadoop/hbase/regionserver/HLog.java
  Made data members private.
  (getFileNum): Added accessor for tests.
M  src/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  Added info to logs..
  (snapshotMemcaches): removed.
  (internalFlushcache): No longer takes startime.  Internally now does
  some of what used happen in snapshotMemcaches including sending of
  message to stores to snapshot.
  (getEntrySize): Added method for calculating size of an update. Used
  by HRegion and flushing so both come up w/ same answer.
M  src/java/org/apache/hadoop/hbase/util/Merge.java
  Add logging of whats happening during merges and fail earlier than we
  used if stuff is not right.
  Renamed local variables from region1 to r1, etc., so didn't clash
  with data members of same name.
M  src/java/org/apache/hadoop/hbase/util/MetaUtils.java
  Added a TODO



git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@650298 13f79535-47bb-0310-9956-ffa450edef68
2008-04-21 22:19:30 +00:00
Michael Stack e411d3c61f HBASE-595 RowFilterInterface.rowProcessed() is called *before* fhe final
filtering decision is made


git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@650202 13f79535-47bb-0310-9956-ffa450edef68
2008-04-21 16:46:31 +00:00
Michael Stack 66e545adf0 HBASE-598 Loggging, no .log file; all goes into .out
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@649938 13f79535-47bb-0310-9956-ffa450edef68
2008-04-20 15:40:07 +00:00
Michael Stack e31af6a288 HBASE-598 Loggging, no .log file; all goes into .out
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@649937 13f79535-47bb-0310-9956-ffa450edef68
2008-04-20 15:32:29 +00:00
Jim Kellerman 628124f7ed HBASE-590 HBase migration tool does not get correct FileSystem or root directory if configuration is not correct.
Commit change to trunk.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@649668 13f79535-47bb-0310-9956-ffa450edef68
2008-04-18 19:47:21 +00:00
Jim Kellerman 5af9719de3 HBASE-10 HRegionServer hangs upon exit due to DFSClient Exception
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@649373 13f79535-47bb-0310-9956-ffa450edef68
2008-04-18 05:32:23 +00:00
Michael Stack ef5bb6f316 HBASE-532 Odd interaction between HRegion.get, HRegion.deleteAll and compactions
Found an issue in the 532 commit.  The MemcacheScanner had a flipped isWildcardScanner
test; fixing it returned wrong columns because using okCols rather than the literals
passed in.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@648912 13f79535-47bb-0310-9956-ffa450edef68
2008-04-17 01:01:50 +00:00
Michael Stack 373761483f HBASE-532 Odd interaction between HRegion.get, HRegion.deleteAll and compactions
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@648850 13f79535-47bb-0310-9956-ffa450edef68
2008-04-16 21:07:32 +00:00
Michael Stack dbd74f2f9a Add note that we bundle junit
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@648816 13f79535-47bb-0310-9956-ffa450edef68
2008-04-16 19:46:21 +00:00
Michael Stack 961597af2a Add LICENSE for junit.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@648814 13f79535-47bb-0310-9956-ffa450edef68
2008-04-16 19:44:37 +00:00
Michael Stack 9ad3122021 HBASE-582 HBase 554 forgot to clear results on each iteration caused by a filter
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@648798 13f79535-47bb-0310-9956-ffa450edef68
2008-04-16 19:23:41 +00:00
Michael Stack ebbdb04cc9 Add to 'getting started' note about hbase being file handles hog.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@648746 13f79535-47bb-0310-9956-ffa450edef68
2008-04-16 16:45:00 +00:00
Jim Kellerman baa84cb4a1 HBASE-575 master dies with stack overflow error if rootdir isn't qualified
git-svn-id: https://svn.apache.org/repos/asf/hadoop/hbase/trunk@648427 13f79535-47bb-0310-9956-ffa450edef68
2008-04-15 21:45:28 +00:00