Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,361 Commits 95 Branches 866 Tags
Commit Graph

4361 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nicolas Spiegelberg
04afdf9633 HBASE-3690 Option to Exclude Bulk Import Files from Minor Compaction 
Summary:
We ran an incremental scrape with HFileOutputFormat and
encountered major compaction storms. This is caused by the bug in
HBASE-3404. The permanent fix is a little tricky without HBASE-2856. We
realized that a quicker solution for avoiding these compaction storms is
to simply exclude bulk import files from minor compactions and let them
only be handled by time-based major compactions. Add with functionality
along with a config option to enable it.

Rewrote this feature to be done on a per-bulkload basis.

Test Plan:
 - mvn test -Dtest=TestHFileOutputFormat

DiffCamp Revision:

Reviewers: stack, Kannan, JIRA, dhruba

Reviewed By: stack

CC: dhruba, lhofhansl, nspiegelberg, stack

Differential Revision: 357

git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1200621 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 23:31:13 +00:00
Nicolas Spiegelberg
6c57076f7e HBASE-4765 Enhance HLog Metrics 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1200619 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 23:29:16 +00:00
Andrew Kyle Purtell
373c98b0df HBASE-4554 Allow set/unset coprocessor table attributes from shell (Mingjie Lai) 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1200436 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 17:15:20 +00:00
Zhihong Yu
076030550f HBASE-4737 Addendum removes unnecessary print 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1200138 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 04:18:42 +00:00
Zhihong Yu
f5944d1a72 HBASE-4737 Addendum tags SmallTests 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1200012 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-09 22:37:32 +00:00
Nicolas Spiegelberg
e896f6d400 Upgrade nextKeyValue() error message from debug to info to highlight MR generation failures 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1199605 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-09 03:48:11 +00:00
Zhihong Yu
1f2aadb80d HBASE-4737 Categorize the tests into small/medium/large; allow small tests to be run 
in parallel within a single JVM (N Keywal)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1199604 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-09 03:38:10 +00:00
Michael Stack
007e7dc183 HBASE-4759 Migrate from JUnit 4.8.2 to JUnit 4.10 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1199476 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-08 21:51:51 +00:00
Michael Stack
ea5f9bf29a HBASE-4756 Enable tab-completion in HBase shell 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1199102 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-08 04:37:27 +00:00
Zhihong Yu
78ebe94d05 HBASE-4740 [bulk load] the HBASE-4552 API can't tell if errors on region server are recoverable 
(Jonathan Hsieh)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1199017 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-07 23:51:56 +00:00
Zhihong Yu
d54b1fdee7 HBASE-4754 FSTableDescriptors.getTableInfoPath() should handle FileNotFoundException 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1198953 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-07 21:40:28 +00:00
Zhihong Yu
f7e925c8d3 HBASE-4746 Use a random ZK client port in unit tests so we can run them in parallel 
(Mikhail Bautin)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1198856 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-07 18:22:57 +00:00
Michael Stack
02f6104dc2 HBASE-4753 org.apache.hadoop.hbase.regionserver.TestHRegionInfo#testGetSetOfHTD throws NPE on trunk 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1198581 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-06 22:47:17 +00:00
Michael Stack
7d8d42d01b HBASE-3939 Some crossports of Hadoop IPC fixes 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1198573 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-06 21:48:31 +00:00
Michael Stack
b508f52998 HBASE-4751 Make TestAdmin#testEnableTableRoundRobinAssignment friendly to concurrent tests 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1198013 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-05 18:53:49 +00:00
Michael Stack
7052e59b38 HBASE-4749 TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS occasionally fails 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1198007 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-05 18:45:24 +00:00
Zhihong Yu
2ee75203ed HBASE-4747 Addendum upgrade maven failsafe 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197972 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-05 15:38:31 +00:00
Andrew Kyle Purtell
c775d60de6 HBASE-4745 LRU statistics thread should be a daemon 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197864 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-05 01:48:45 +00:00
Michael Stack
56e19a72dc HBASE-4725 NPE in AM#updateTimers 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197815 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-04 23:10:08 +00:00
Michael Stack
bb0c9a11d8 HBASE-4553 The update of .tableinfo is not atomic; we remove then rename 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197812 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-04 23:03:29 +00:00
Zhihong Yu
c824695af9 HBASE-4747 Upgrade maven surefire plugin to 2.10 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197785 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-04 21:22:55 +00:00
Michael Stack
2ce1d7c97c HBASE-4722 TestGlobalMemStoreSize has started failing; ADDENDUM 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197349 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 23:39:48 +00:00
Doug Meil
ae6af9e630 HBASE-4743 book.xml, performance.xml, troubleshooting.xml scan info 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197315 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 21:22:57 +00:00
Michael Stack
92b57170e9 HBASE-4298 Support to drain RS nodes through ZK 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1197256 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 18:34:41 +00:00
Gary Helmling
a9992a53c4 HBASE-4518 TestServerCustomProtocol fails intermittently 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196952 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 05:57:36 +00:00
Michael Stack
09225cf9b6 HBASE-4480 Testing script to simplify local testing 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196939 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 04:11:46 +00:00
Michael Stack
0c07e905a2 HBASE-4480 Testing script to simplify local testing 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196938 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 04:08:42 +00:00
Michael Stack
f24e30f5e7 HBASE-4719 HBase script assumes pre-Hadoop 0.21 layout of jar files 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196936 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 04:05:37 +00:00
Doug Meil
5eaea899e3 HBASE-4736 troubleshooting.xml, xrefs to Arch chapter 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196925 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 03:05:37 +00:00
Zhihong Yu
76060691bb HBASE-4609 ThriftServer.getRegionInfo() is expecting old ServerName format, need to 
use new Addressing class instead (Jonathan Gray)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196873 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 23:48:22 +00:00
Doug Meil
c61f0f296f HBASE-4735 book.xml, schema design keysize code example 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196860 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 23:26:30 +00:00
Doug Meil
88643f7220 HBASE-4732 book.xml, added link to MR tutorial in MR chapter (trivial change) 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196805 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 21:19:13 +00:00
Doug Meil
b08f746cd2 HBASE-4731 book.xml, schema design - rowkey numeric example 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196801 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 21:09:47 +00:00
Doug Meil
f0444014b8 HBASE-4730 book.xml, ops_mgt.xml, performance.xml - handful of changes 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196792 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 20:40:14 +00:00
Nicolas Spiegelberg
b44e09085c HBASE-2312 [jira] Possible data loss when RS goes into GC pause while rolling 
HLog

Summary:
There is a very corner case when bad things could happen(ie data loss):

1)	RS #1 is going to roll its HLog - not yet created the new one, old one will
get no more writes
2)	RS #1 enters GC Pause of Death
3)	Master lists HLog files of RS#1 that is has to split as RS#1 is dead, starts
splitting
4)	RS #1 wakes up, created the new HLog (previous one was rolled) and appends an
edit - which is lost

The following seems like a possible solution:

1)	Master detects RS#1 is dead
2)	The master renames the /hbase/.logs/<regionserver name>  directory to
something else (say /hbase/.logs/<regionserver name>-dead)
3)	Add mkdir support (as opposed to mkdirs) to HDFS - so that a file create
fails if the directory doesn't exist. Dhruba tells me this is very doable.
4)	RS#1 comes back up and is not able create the new hlog. It restarts itself.

Test Plan: EMPTY
Reviewers: JIRA, stack, khemani
Reviewed By: khemani
CC: tedyu, nspiegelberg, stack, Kannan, khemani, jgray
Differential Revision: 99

git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196773 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 19:24:23 +00:00
Nicolas Spiegelberg
119ae0c4ac [jira] [HBASE-4627] Ability to specify a custom start/end to RegionSplitter 
Summary:
[HBASE-4627]

added a custom start/end row to RegionSplitter.  Also solved
an off-by-one error because the end row is prefix-inclusive and not
exclusive.

<a href="https://issues.apache.org/jira/browse/HBASE-4489" title="Better key splitting in RegionSplitter"><del>HBASE-4489</del></a> changed the default endKey on HexStringSplit from 7FFF... to FFFF...  While this is correct, existing users of 0.90 RegionSplitter have 7FFF as the end key in their schema and the last region will not split properly under this new code.  We need to let the user specify a custom start/end key range for when situations like this arise.  Optimally, we should also write the start/end key in META so we could figure this out implicitly instead of requiring the user to explicitly specify it.

Test Plan:
 - mvn test -Dtest=TestRegionSplitter

CC: JIRA

Reviewers: DUMMY_REVIEWER

Differential Revision: 39

git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196772 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 19:24:18 +00:00
Zhihong Yu
5721a5ac53 HBASE-4716 Addendum Use Preconditions and remove enum 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196749 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 18:35:52 +00:00
Zhihong Yu
0e32567cdd HBASE-4716 Improve locking for single column family bulk load 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196674 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 16:28:09 +00:00
Michael Stack
321d6c3025 HBASE-4724 TestAdmin hangs randomly in trunk 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196636 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 15:37:19 +00:00
Michael Stack
d6601d1d17 HBASE-4713 Raise debug level to warn on ExecutionException in HConnectionManager 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196592 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 14:15:54 +00:00
Zhihong Yu
87a157d9eb HBASE-1744 HBaseAdmin ctor should obtain Configuration from HBaseTestingUtility 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196533 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 10:12:24 +00:00
Zhihong Yu
061c22b302 SE-4377 [hbck] Offline rebuild .META. from fs data only 
(Jonathan Hsieh)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196527 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 09:51:19 +00:00
Michael Stack
3fab67a8b5 HBASE-4722 TestGlobalMemStoreSize has started failing; commit some extra logging to help debug whats going on up on jenkins 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196464 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 06:12:38 +00:00
Zhihong Yu
1b63b76e69 HBASE-1744 Thrift server to match the new java api (Tim Sell) 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196451 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 04:26:26 +00:00
Jean-Daniel Cryans
930eb49759 HBASE-3515 [replication] ReplicationSource can miss a log after RS comes out of GC 
(fixing a missing handling of IOE in Replication)


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196402 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-02 00:05:55 +00:00
Jean-Daniel Cryans
08a9170236 HBASE-3515 [replication] ReplicationSource can miss a log after RS comes out of GC 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196398 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 23:44:02 +00:00
Michael Stack
850bb134a6 HBASE-3680 Publish more metrics about mslab; REVERT 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196389 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 23:19:59 +00:00
Michael Stack
e44d373842 HBASE-4708 Revert safemode related pieces of hbase-4510 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196321 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 22:10:10 +00:00
Nicolas Spiegelberg
86468b381f Fixed CHANGES file for HBASE-4532 & HBASE-4611 
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196288 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 21:26:16 +00:00
Nicolas Spiegelberg
8742f64536 Revert "HBASE-4627 Ability to specify a custom start/end to RegionSplitter" 
This reverts commit r1196256.


git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1196258 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 20:25:08 +00:00