Commit Graph

3052 Commits

Author SHA1 Message Date
Michael Stack 5affba7171 HBASE-2641 Refactor HLog splitLog, hbase-2437 continued; break out split code as new classes
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001969 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 23:52:46 +00:00
Michael Stack 7faaf57d1b HBASE-3030 The return code of many filesystem operations are not checked
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001954 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 23:02:52 +00:00
Michael Stack b11ac8d6f5 HBASE-3040 BlockIndex readIndex too slowly in heavy write scenario
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001949 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 22:53:26 +00:00
Jonathan Gray 4f545ea09c Attempting to fix TestHeapSize on Hudson (align array heapsize)
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001944 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 22:43:59 +00:00
Michael Stack 9069ed1ac6 HBASE-2995 Incorrect dependency on Log class from Jetty
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001936 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 22:18:28 +00:00
Michael Stack aff46efe9e HBASE-3042 Use LO4J in SequenceFileLogReader
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001928 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 22:09:51 +00:00
Michael Stack 823cfeac9c HBASE-2641 Refactor HLog splitLog, hbase-2437 continued; break out split code as new classes
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001924 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 22:02:23 +00:00
Michael Stack 10a84f3b88 HBASE-3039 Stuck in regionsInTransition because rebalance came in at same time as a split
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  Add region name to warning log message (w/o it message is no good).
M src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
  Add src of split message else need to deduce where it came from by looking
  elsewhere.
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
  Updated log messages to include region and where appropritate source
  server name; debug hard w/o
  Changed regionOnline and regionOffline to check for unexpected
  states and log warnings rather than proceed regardless.
  Added in fix for concurrent balance+split; split message now
  updates regionsintransition where previous it did not.
  Remove checkRegion method.  Its a reimplementation of
  what regionOnline and regionOffline do only less comprehensive
  regards what gets updated (this.regions + this.servers rather
  than this.regions, this.servers and regionsInTransition)
  That they were less comprehensive is root of this bug.
M src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  Make the message about why we are not running balancer richer
  (print out how many reigons in transition and more of the
  regionsintrnasition list).
M src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java
  Javadoc and minor formatting.


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001911 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 21:02:04 +00:00
Michael Stack 5efa0ba9c9 Added note on hlog tool, that it can be used to look at files in recovered edits file
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001907 13f79535-47bb-0310-9956-ffa450edef68
2010-09-27 20:58:26 +00:00
Michael Stack 1339f395d6 HBASE-3037 When new master joins running cluster does "Received report from unknown server -- telling it to STOP_REGIONSERVER..."
M src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 On regionServerReport, if we get a report from an 'unknown' regionserver,
 we used to tell it stop itself.  Now, if 'unknown' server AND
 this master did not start the cluster, its joining the cluster, then
 treat the report as a regionServerStart and register the incoming
 server rather than tell it shutdown.
M src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Pass the freshClusterStartup flag to ServerManager.
 Add more executors for opening and closing.  On cluster startup a
 master shouldn't be bottleneck clearning the server opens.
 Expose the run-balancer method so can make it available in
 ServeAdmin.
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
 Minor formatting and javadoc
M src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPCProtocolVersion.java
 Upped rpc version number because of new balancer addition (and because
 we didn't do it when we put in new master).
M src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java
 Added balance method.
M src/main/resources/hbase-default.xml
 Change how ofter we check in from every 3 seconds to every 5 seconds.


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001140 13f79535-47bb-0310-9956-ffa450edef68
2010-09-25 06:24:42 +00:00
Michael Stack e58ebc4830 HBASE-3036 avro tests failing up on hudson (pass locally) -- trying something, passing Configuration to HBaseImpl
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001062 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 20:40:41 +00:00
Jean-Daniel Cryans d4e9b8f4d9 HBASE-3033 [replication] ReplicationSink.replicateEntries improvements
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001061 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 20:37:44 +00:00
Jean-Daniel Cryans b05d9ac937 HBASE-2992 [replication] MalformedObjectNameException in ReplicationMetrics
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001016 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 18:34:42 +00:00
Jean-Daniel Cryans da611388b4 HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1001014 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 18:32:26 +00:00
Michael Stack 4163b90a87 Fix broke TestHLogSplit test
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000966 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 16:59:42 +00:00
Michael Stack d988546130 Fix broke TestHLogSplit test
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000965 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 16:59:17 +00:00
Michael Stack 9ea9cba5b2 HBASE-3028 No basescanner means no GC'ing of split, offlined parent regions
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000715 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 05:46:41 +00:00
Todd Lipcon 2ecdd83e0d HBASE-3002. Fix zookeepers.sh to work properly with strange JVM options
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000698 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 04:10:55 +00:00
Michael Stack a85ee11f3b HBASE-2889 Tool to look at HLogs -- parse and tail -f; fix for suppression of EOFEs by Nicolas
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000697 13f79535-47bb-0310-9956-ffa450edef68
2010-09-24 04:00:48 +00:00
Ryan Rawson 0f3c62e19b HBASE-2450 For single row reads of specific columns, seek to the first column in HFiles rather than start of (Pranav via Ryan, some Ryan)
HBASE-2916  Reseek directly to next column
HBASE-2959  Scanning always starts at the beginning of a row



git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000276 13f79535-47bb-0310-9956-ffa450edef68
2010-09-22 23:51:55 +00:00
Ryan Rawson 2806d60cab HBASE-3003 ClassSize constants dont use 'final'
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000275 13f79535-47bb-0310-9956-ffa450edef68
2010-09-22 23:49:49 +00:00
Michael Stack 17da1d0369 HBASE-3001 Ship dependency jars to the cluster for all jobs
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000261 13f79535-47bb-0310-9956-ffa450edef68
2010-09-22 22:16:23 +00:00
Michael Stack a46c74894f HBASE-3026 Fixup of missing daughters on split is too aggressive
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000133 13f79535-47bb-0310-9956-ffa450edef68
2010-09-22 18:42:41 +00:00
Michael Stack 0de40fe1b0 Inserted an email Ryan wrote the list on 'considerations sizing regions'
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1000113 13f79535-47bb-0310-9956-ffa450edef68
2010-09-22 18:05:32 +00:00
Michael Stack af3bf8cad0 HBASE-3024 NPE processing server crash in MetaEditor.addDaughter
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999672 13f79535-47bb-0310-9956-ffa450edef68
2010-09-21 22:24:05 +00:00
Michael Stack 5fa3d69f66 HBASE-3023 NPE processing server crash in MetaReader. getServerUserRegions
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999664 13f79535-47bb-0310-9956-ffa450edef68
2010-09-21 22:00:12 +00:00
Michael Stack ffedf2c58e HBASE-3022 Change format of enum messages in o.a.h.h.executor package
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999660 13f79535-47bb-0310-9956-ffa450edef68
2010-09-21 21:44:21 +00:00
Michael Stack 14d8fb2ab7 HBASE-3018 Bulk assignment on startup runs serially through the cluster servers assigning in bulk to one at a time
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
  Removed reentrant lock that spanned moving region into
  regionsintransition and the getting of lock on state object.  It
  was a bad idea.  I found it actually deadlocked.  It shouldn't be needed.
  The call to assign will fail if the state instance is not of the right
  'state' type.
  Run a thread per server parcelling out region assignments so we assign
  across the cluster concurrently rather than in series as we were doing.


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999237 13f79535-47bb-0310-9956-ffa450edef68
2010-09-21 06:36:22 +00:00
Michael Stack 34f17faa5d HBASE-3017 More log pruning
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999154 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 22:43:25 +00:00
Michael Stack 4f0d6bc6be HBASE-3017 More log pruning
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999150 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 22:31:19 +00:00
Michael Stack 0cdd2c5eb7 HBASE-3017 More log pruning
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999143 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 22:25:54 +00:00
Michael Stack 1456bfff76 HBASE-3000 Add "hbase classpath" command to dump classpath
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999062 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 19:14:48 +00:00
Michael Stack 98b9448162 HBASE-3000 Add "hbase classpath" command to dump classpath
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999060 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 19:13:43 +00:00
Michael Stack a3856ca945 HBASE-3017 More log pruning
M conf/log4j.properties
  Make ZKW log at INFO-level
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
  Log message cleanup.
M src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  Remove excessive hostname+port qualifier on master for zk messages
M src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
  Log message cleanup
M src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
  Format the ServerMonitor message.
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  Remove excessive hostname on zk message id; just add port
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
  Cleanup of messages.


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999057 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 19:04:41 +00:00
Michael Stack 7daffb3a42 HBASE-3015 recovered.edits files not deleted if it only contain edits that have already been flushed; hurts perf for all future opens of the region
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999046 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 18:43:13 +00:00
Michael Stack 3a6705b08b TestCompaction was hanging because it was blocking... up the multiplier so we don't block any more
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@999017 13f79535-47bb-0310-9956-ffa450edef68
2010-09-20 17:35:05 +00:00
Michael Stack 467a0e726d HBASE-3016 Refactor zk logging in trunk; do less and and identifier to log messages to help debugging
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@998555 13f79535-47bb-0310-9956-ffa450edef68
2010-09-18 21:56:37 +00:00
Michael Stack 2dfce22970 Another attempt at fixing avro test -- I had time out for test too short
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@998552 13f79535-47bb-0310-9956-ffa450edef68
2010-09-18 21:32:35 +00:00
Michael Stack 46c53f1460 Fix failing avro test
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@998504 13f79535-47bb-0310-9956-ffa450edef68
2010-09-18 18:06:22 +00:00
Michael Stack d153ec95da HBASE-3010 Can't start/stop/start... cluster using new master
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@998380 13f79535-47bb-0310-9956-ffa450edef68
2010-09-18 00:51:05 +00:00
Michael Stack e532293310 HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls severly impacting performance--Now add fix I intended, a spelling mistake in HFile
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997975 13f79535-47bb-0310-9956-ffa450edef68
2010-09-17 04:21:28 +00:00
Michael Stack 96f8cbe9e2 HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls severly impacting performance--Revert mistaken over commit
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997974 13f79535-47bb-0310-9956-ffa450edef68
2010-09-17 04:20:55 +00:00
Michael Stack cac06d4573 HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls severly impacting performance
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997973 13f79535-47bb-0310-9956-ffa450edef68
2010-09-17 04:18:33 +00:00
Michael Stack 763865630c HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls severly impacting performance
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997968 13f79535-47bb-0310-9956-ffa450edef68
2010-09-17 03:56:16 +00:00
Michael Stack eab3e07e39 Fix hanging TestCompaction test -- it was putting up the blocking gate and never moving on
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997589 13f79535-47bb-0310-9956-ffa450edef68
2010-09-16 05:09:10 +00:00
Michael Stack 5145547154 HBASE-2899 hfile.min.blocksize.size ignored/documentation wrong
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997544 13f79535-47bb-0310-9956-ffa450edef68
2010-09-15 23:57:15 +00:00
Ryan Rawson 5b600805b5 HBASE-2782 QOS for META table access
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997541 13f79535-47bb-0310-9956-ffa450edef68
2010-09-15 23:44:08 +00:00
Michael Stack 191a334487 Rename TestReplicationSink as DISABLEDTestReplicationSink till fixed
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997539 13f79535-47bb-0310-9956-ffa450edef68
2010-09-15 23:26:45 +00:00
Ryan Rawson 55633e79f1 HBASE-2997 Performance fixes - profiler driven (build/test fix)
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997533 13f79535-47bb-0310-9956-ffa450edef68
2010-09-15 22:50:49 +00:00
Ryan Rawson 009f461738 HBASE-2997 Performance fixes - profiler driven
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@997519 13f79535-47bb-0310-9956-ffa450edef68
2010-09-15 21:48:04 +00:00