Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,310 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

5126 Commits

Author SHA1 Message Date
zhangduo ff23e15769 HBASE-18552 Backport the server side change in HBASE-18489 to branch-1 2017-10-12 10:42:21 +08:00
tedyu 2828427219 HBASE-18959 Backport HBASE-18874 (HMaster abort message will be skipped if Throwable is passed null) to branch-1 (Pankaj Kumar) 2017-10-10 08:33:42 -07:00
Sean Busbey aa50971947 Revert "HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned" 
This reverts commit 256fc63007.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2017-10-09 17:06:14 -05:00
Sean Busbey c7dc0da849 Revert "HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT" 
This reverts commit ead1063244.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2017-10-09 17:06:09 -05:00
Peter Somogyi 8d77c1e954 HBASE-18957 add test that confirms 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will return results that match either of the FamilyFilters 
Amending-Author: Sean Busbey <busbey@apache.org>

Signed-off-by: Michael Stack <stack@apache.org>
 2017-10-09 17:05:39 -05:00
Maytee Chinavanichkit 50ef066c89 HBASE-18921 Fix Result.current ArrayIndexOutOfBoundsException 
Patch ArrayIndexOutOfBoundsException when current() is called after
advance() has already returned false

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-10-08 02:58:01 +08:00
Yi Liang fc783ef045 HBASE-16894 Create more than 1 split per region, generalize HBASE-12590 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-10-03 17:24:45 -07:00
libisthanks 1c7321f9d5 HBASE-18814 Make ScanMetrics enabled and add counter <HBase Counters, ROWS_SCANNED> into the MapReduce Job over snapshot 
Signed-off-by: Ashu Pachauri <ashu@apache.org>
 2017-10-02 18:25:52 -07:00
Pankaj Kumar b2943504b8 HBASE-18890, Backport HBASE-14499 (Master coprocessors shutdown will not happen on master abort) to branch-1 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-29 16:20:52 -07:00
Andrew Purtell 93ad1aba70 HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 
Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicasClient.java
 2017-09-29 15:43:03 -07:00
shaofengshi aa86657a52 HBASE-18885 HFileOutputFormat2 hardcodes default FileOutputCommitter 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-27 06:58:43 -07:00
Andrew Purtell 3abc0458e9 Amend HBASE-18830 TestCanaryTool does not check Canary monitor's error code 
Adjust exception control flow to fix findbugs warning
NP_NULL_ON_SOME_PATH_EXCEPTION, Possible null pointer dereference of
regionSink in org.apache.hadoop.hbase.tool.Canary$RegionMonitor.run()
on exception path
 2017-09-26 10:38:09 -07:00
Michael Stack eeaa9aee66 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (Biju Nair) 2017-09-25 20:51:11 -07:00
Chinmay Kulkarni 9d68535e8e HBASE-18830 TestCanaryTool does not check Canary monitor's error code 
Added assertion checks to make sure that the error code for the
ToolRunner run() method is used.

Testing Done: Checked that TestCanaryTool unit tests fail when there is
an error code in the current Canary run.

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 2017-09-25 19:17:22 -07:00
Chinmay Kulkarni 414e0ca292 HBASE-18762 Canary sink type cast error 
Changed the type hierarchy of Canary sinks to reduce confusion and avoid
cast errors.

Testing Done: Ran the TestCanaryTool.java test suite and confirmed that
the working is correct.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2017-09-25 19:17:22 -07:00
Andrew Purtell d74bdc3100 Amend HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas 
Remove now invalid unit test TestCorruptedRegionStoreFile
 2017-09-25 17:36:54 -07:00
Andrew Purtell 5a3d9cb225 HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas 2017-09-22 12:41:03 -07:00
Zach York bafc6e523e HBASE-18478 Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible 
This provides significant cluster start time reduction for FileSystems which do not surface locality (S3).

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
 2017-09-19 16:30:43 -07:00
Abhishek Singh Chouhan 2845ddaf9e HBASE-18796 Admin#isTableAvailable returns incorrect result before daughter regions are opened 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-09-19 13:52:15 -07:00
tedyu ead1063244 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-09-18 14:31:21 -07:00
andrewcheng 815673f7e4 HBASE-18131 Add an hbase shell command to clear deadserver list in ServerManager 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-16 09:33:08 -07:00
Biju Nair 7dc0a63c06 HBASE-18641 Include block content verification logic used in lruCache in bucketCache 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-16 06:12:43 -07:00
Andrew Purtell e5f80f36e2 Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4 
Disable units where argument matching is intermittently failing
 2017-09-15 10:42:04 -07:00
Andrew Purtell 469d6bf457 Revert "Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4" 
This reverts commit aa7c93c659.
 2017-09-15 10:42:04 -07:00
Reid Chan 862a717e37 HBASE-18801 Bulk load cleanup may falsely deem file deletion successful 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-15 02:15:06 -07:00
Andrew Purtell aa7c93c659 Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4 
Skip tests where mocking with custom matchers fails unexpectedly on
older JREs.
 2017-09-14 14:38:56 -07:00
Michael Stack 91422357f2 HBASE-18818 TestConnectionImplemenation fails 2017-09-14 12:31:01 -07:00
Andrew Purtell 0621486620 HBASE-18346 TestRSKilledWhenInitializing failing on branch-1 (branch-1.4) 
Temporarily disabled flaky test TestRSKilledWhenInitializing
 2017-09-13 14:48:36 -07:00
Andrew Purtell 3f785a83b2 Revert "HBASE-16488 Starting namespace and quota services in master startup asynchronizely (Stephen Yuan Jiang)" 
This reverts commit af359d03b5.
 2017-09-13 13:24:58 -07:00
Guangxu Cheng a8ac84f1a6 HBASE-18789 Displays the reporting interval of each RS on the Master page 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-13 09:39:09 -07:00
Abhishek Singh Chouhan 432ca7e3fb HBASE-18771 Incorrect StoreFileRefresh leading to split and compaction failures 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-09-12 18:08:50 -07:00
tedyu b6a12c843e HBASE-18652 Expose individual cache stats in a CombinedCache through JMX - revert due to test failure 2017-09-08 14:21:30 -07:00
tedyu ab8a667c45 HBASE-18641 Include block content verification logic used in lruCache in bucketCache - revert due to test failure 2017-09-08 12:45:47 -07:00
Biju Nair a6dbeba22a HBASE-18641 Include block content verification logic used in lruCache in bucketCache 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-08 12:07:02 -07:00
Biju Nair 5bfe1da984 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-06 13:30:44 -07:00
Biju Nair 5de4a7d898 HBASE-18737 Display configured max size of cache on RS UI 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-06 11:06:27 -07:00
Zach York 4733aa4655 HBASE-18757 Fix improper bitwise & in bucketcache offset calculation 
This correctly casts the operand to a long to avoid negative offsets created by sign extending the integer operand.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-05 15:10:08 -07:00
Chun-Hao Tang 8eacd32beb HBASE-18746 Throw exception with job.getStatus().getFailureInfo() when ExportSnapshot fails 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-09-05 02:04:09 +08:00
tedyu d5033fd350 HBASE-18743 HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default table are deleted (wenbang) 2017-09-04 07:52:57 -07:00
Cesar Delgado 8178305d4e HBASE-18675 Making {max,min}SessionTimeout configurable for MiniZooKeeperCluster 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-08-30 14:32:37 -07:00
Sean Mackrory aec4bf6bae HBASE-15947 Classes used only for tests included in main code base 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-08-30 10:53:31 +08:00
Vikas Vishwakarma f071e56a9e HBASE-18633 Add more info to understand the source/scenario of large batch requests exceeding threshold 2017-08-28 15:30:40 +05:30
Ajay Jadhav 7d605fe9c2 HBASE-18448 Added refresh HFiles coprocessor endpoint 
Signed-off-by: Michael Stack <stack@apache.org>
 2017-08-24 16:05:04 -07:00
Mike Drob 362a2924d2 HBASE-18628 Fix event pre-emption in ZKPermWatcher 
Instead of using an Atomic Reference to data and aborting when we detect
that new data comes in, use the native cancellation/pre-emption features
of Java Future.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-08-23 16:42:33 -07:00
Biju Nair 19a80c8234 HBASE-18532 Improve cache related stats rendered on RS UI 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-23 13:26:10 -07:00
Biju Nair ad22437d05 HBASE-18614 Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-23 10:51:27 -07:00
Michael Stack 338232f81f Revert "HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable" 
Lots of tests timing out. Revert for now.

This reverts commit 7a96682643.
 2017-08-22 08:31:55 -07:00
Michael Stack b8857b0a7b HBASE-16615 Fix flaky TestScannerHeartbeatMessages (Duo Zhang) 2017-08-21 16:01:26 -07:00
Andrew Purtell 19fffacd3a HBASE-18623 Frequent failed to parse at EOF warnings from WALEntryStream 2017-08-21 09:29:20 -07:00
Pankaj Kumar 7a96682643 HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable 
Signed-off-by: Michael Stack <stack@apache.org>
 2017-08-18 14:23:59 -07:00
Chia-Ping Tsai f61f09cdf2 HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to next column 2017-08-19 02:14:37 +08:00
Chia-Ping Tsai 10d65ee37f HBASE-18572 Delete can't remove the cells which have no visibility label 2017-08-19 01:56:29 +08:00
tedyu 64bef67558 HBASE-18617 FuzzyRowKeyFilter should not modify the filter pairs (vinisha) 2017-08-18 10:02:06 -07:00
Pankaj Kumar deeda60c69 HBASE-17617 Backport HBASE-16731 (Inconsistent results from the Get/Scan if we use the empty FilterList) to branch-1 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-08-17 18:31:57 +08:00
Zach York 1f1ab8c873 HBASE-18587 Fix flaky TestFileIOEngine 
This short circuits reads and writes with 0 length and also removes flakiness in TestFileIOEngine

Signed-off-by: Michael Stack <stack@apache.org>
 2017-08-16 14:50:11 -07:00
Andrew Purtell 3feb87b005 HBASE-18431 Mitigate compatibility concerns between branch-1.3 and branch-1.4 
- Refactor TableName into its own proto module and fix up users

- Move SnapshotDescription from Snapshot.proto back to HBase.proto

- Restore FastLongHistogram and TestFastLongHistogram;
  deprecate FastLongHistogram

- Move DeleteQueryTracker back to o.a.h.h.regionserver
 2017-08-16 11:34:18 -07:00
Ramkrishna 6255dc7001 HBASE-18251 Remove unnecessary traversing to the first and last keys in 
the CellSet (Toshihoro Suzuki)
 2017-08-16 10:50:58 -07:00
Ashish Singhi a49d43bfbf HBASE-18437 Revoke access permissions of a user from a table does not work as expected 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 2017-08-15 18:59:21 -07:00
Pankaj Kumar d7c6a0bf43 HBASE-18512, Region Server will abort with IllegalStateException if HDFS umask has limited scope 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-15 13:26:28 -07:00
tedyu 3ed765257e HBASE-18526 FIFOCompactionPolicy pre-check uses wrong scope (Vladimir Rodionov) 2017-08-15 11:09:55 -07:00
Zach York d6a781cf08 HBASE-18533 Expose BucketCache values to be configured 
Before this commit, BucketCache always used the default values.
This commit adds the ability to configure these values.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-14 13:28:42 -07:00
Jerry He 9078a034c4 HBASE-18522 Add RowMutations support to Batch 2017-08-14 09:21:53 -07:00
Esteban Gutierrez 578e29f96b HBASE-18025 CatalogJanitor should collect outdated RegionStates from the AM 2017-08-11 14:39:16 -05:00
Jan Hentschel b181f172ef HBASE-18197 Avoided to call job.waitForCompletion(true) two times 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-08-12 02:47:25 +08:00
Yu Li a03c2036e9 HBASE-18469 Correct RegionServer metric of totalRequestCount 2017-08-11 14:53:23 +08:00
Esteban Gutierrez 15bad3c036 HBASE-18024 HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists 2017-08-10 18:54:01 -05:00
Ashu Pachauri cab492d34f HBASE-18398: Snapshot operation fails with FileNotFoundException 2017-08-10 14:49:59 -07:00
Andrew Purtell 685ab19064 HBASE-18248 Warn if monitored RPC task has been tied up beyond a configurable threshold 2017-08-09 18:17:21 -07:00
Zach York 036fce3cf8 HBASE-18520 Add jmx value to determine true Master Start time; amendment 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-08 15:56:56 +08:00
Michael Stack b5a24b6696 Revert "HBASE-18520 Add jmx value to determine true Master Start time" 
This reverts commit 1ba949e030.
 2017-08-08 15:43:58 +08:00
Zach York 3b49e60e17 HBASE-18520 Add jmx value to determine true Master Start time 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-08 15:27:19 +08:00
Michael Stack ce91ab4871 Revert "HBASE-18520 Add jmx value to determine true Master Start time" 
This reverts commit 1ba949e030.
 2017-08-08 15:25:07 +08:00
Zach York 1ba949e030 HBASE-18520 Add jmx value to determine true Master Start time 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-08 15:02:58 +08:00
Zach York cad3d55892 HBASE-18520 Add jmx value to determine true Master Start time 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-05 22:34:26 -07:00
Chia-Ping Tsai a35dc0dddf HBASE-18480 The cost of BaseLoadBalancer.cluster is changed even if the rollback is done 2017-08-02 08:41:53 +08:00
James Taylor 80b85a857c HBASE-18487 Minor fixes in row lock implementation 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-08-01 11:28:28 -07:00
Abhishek Singh Chouhan abb9d88dce HBASE-18374 RegionServer Metrics improvements 2017-07-31 12:47:11 +05:30
Guanghao Zhang 59cad7d3e3 HBASE-18481 The autoFlush flag was not used in PE tool 2017-07-31 11:12:33 +08:00
Abhishek Singh Chouhan 9278037108 HBASE-15134 Add visibility into Flush and Compaction queues 2017-07-28 13:21:04 +05:30
Esteban Gutierrez e662d04321 HBASE-18362 hbck should not report split replica parent region from meta as errors (Huaxiang Sun) 
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
 2017-07-27 15:59:07 -05:00
Mike Drob aca8a124c0 HBASE-17922 Clean TestRegionServerHostname for hadoop3. 
Change-Id: I6f1514b1bc301be553912539e6a4192c2ccc782b
Signed-off-by: Apekshit Sharma <appy@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerHostname.java
 2017-07-26 15:27:58 -07:00
Andrew Purtell a4e1af1fcc HBASE-18054 log when we add/remove failed servers in client (Ali) 
Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/FailedServers.java
 2017-07-25 18:53:01 -07:00
huaxiang sun 9427d6c73c HBASE-18247 Hbck to fix the case that replica region shows as key in the meta table 
Change-Id: I2169290ec5a97a63747bc93ee37f36729c3ac916
 2017-07-25 17:08:53 -07:00
Josh Elser c7e1d67df7 HBASE-18023 Update row threshold warning from 1k to 5k (addendum) 2017-07-25 18:39:44 -04:00
rgidwani 26247996d2 HBASE-15816 Provide client with ability to set priority on Operations 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-07-21 17:12:25 -07:00
anoopsamjohn 9036556a33 HBASE-16993 BucketCache throw java.io.IOException: Invalid HFile block magic when configuring hbase.bucketcache.bucket.sizes. 2017-07-20 23:00:48 +05:30
Andrew Purtell cfd5b6b59f HBASE-18330 NPE in ReplicationZKLockCleanerChore 2017-07-19 15:46:45 -07:00
Chia-Ping Tsai 2da5b432a1 HBASE-18308 Eliminate the findbugs warnings for hbase-server 2017-07-20 00:36:16 +08:00
Phil Yang 980209579b HBASE-18390 Sleep too long when finding region location failed 2017-07-19 12:05:21 +08:00
Stephen Yuan Jiang af359d03b5 HBASE-16488 Starting namespace and quota services in master startup asynchronizely (Stephen Yuan Jiang) 2017-07-18 06:58:29 -07:00
tedyu 5a28437a74 HBASE-18377 Error handling for FileNotFoundException should consider RemoteException in openReader() 2017-07-18 06:50:02 -07:00
tedyu e1bf966f93 HBASE-18358 Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish' 2017-07-11 18:12:15 -07:00
Chia-Ping Tsai bec34ae432 HBASE-18295 The result contains the cells across different rows 2017-07-12 02:46:13 +08:00
Chia-Ping Tsai 44651e52d8 HBASE-18267 The result from the postAppend is ignored 2017-07-11 10:31:21 +08:00
tedyu 5d710bbe7f HBASE-18312 Ineffective handling of FileNotFoundException in FileLink.tryOpen() 2017-07-05 21:02:28 -07:00
Andrew Purtell 3dd55fa0c0 Set versions on branch-1 to 1.5.0-SNAPSHOT 2017-07-03 18:01:15 -07:00
Phil Yang 3381c6c453 HBASE-17931 Assign system tables to servers with highest version 2017-07-03 11:35:31 +08:00
David Harju 743f454d25 HBASE-18023 Log multi-* requests for more than threshold number of rows 
Signed-off-by: Josh Elser <elserj@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2017-06-30 18:45:36 -07:00
Sean Busbey 9264e8fb17 HBASE-17995 improve log messages during snapshot tests. 
Signed-off-by: Michael Stack <stack@apache.org>

 Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
 2017-06-30 10:16:05 -05:00
zhangduo 67b61e6d5a HBASE-16585 Rewrite the delegation token tests with Parameterized pattern 2017-06-30 16:26:34 +08:00
Kahlil Oppenheimer bb181ff50b HBASE-18164 Fast locality computation in balancer - addendum handles NaN 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-06-27 13:33:48 -05:00
Sean Busbey 386a6e87fe Revert "HBASE-18164 Fast locality computation in balancer - addendum handles NaN" 
This reverts commit 74e9d31474.

early commit missed some review feedback.
 2017-06-27 13:33:23 -05:00
Kahlil Oppenheimer 74e9d31474 HBASE-18164 Fast locality computation in balancer - addendum handles NaN 
-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.

-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer

-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-26 12:40:30 -07:00
Pankaj Kumar d5bba50794 HBASE-18167 OfflineMetaRepair tool may cause HMaster to abort always 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-22 02:11:07 -07:00
tedyu 3d9c544428 HBASE-18167 OfflineMetaRepair tool may cause HMaster to abort always - revert wrong version of patch 2017-06-22 02:10:25 -07:00
tedyu 940f4107b0 HBASE-18226 Disable reverse DNS lookup at HMaster and use the hostname provided by RegionServer (Duo Xu) 2017-06-21 18:23:30 -07:00
Stephen Yuan Jiang 105c5c36e6 HBASE-15691 ConcurrentModificationException in BucketAllocator 2017-06-21 15:23:59 -07:00
QilinCao 7b977695aa HBASE-18252 Resolve BaseLoadBalancer bad practice warnings 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-21 10:13:18 -07:00
Pankaj Kumar 01027f805b HBASE-18167 OfflineMetaRepair tool may cause HMaster to abort always 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-21 08:22:30 -07:00
Stephen Yuan Jiang 532e0dda16 HBASE-18036 Data locality is not maintained after cluster restart or SSH (Stephen Yuan Jiang) 2017-06-20 13:07:19 -07:00
Ashish Singhi dead08d668 HBASE-18212 reduce log level for unbuffer warning. 
In Standalone mode with local filesystem HBase logs Warning message:Failed to invoke 'unbuffer' method in class org.apache.hadoop.fs.FSDataInputStream

Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-06-20 02:18:44 -05:00
Kahlil Oppenheimer 4a2f475ed3 HBASE-18164 Fast locality computation in balancer 
-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.

-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer

-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
 2017-06-20 02:18:43 -05:00
Pankaj Kumar d5749bf8ed HBASE-18180 Possible connection leak while closing BufferedMutator in TableOutputFormat 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-19 06:39:27 -07:00
Andrew Purtell 316e02e3d8 HBASE-18219 Fix typo in constant HConstants.HBASE_CLIENT_MEAT_REPLICA_SCAN_TIMEOUT 2017-06-14 16:03:54 -07:00
huzheng 256fc63007 HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-06-13 20:28:04 +08:00
zhangduo d6e85b0511 Revert "FilterList with MUST_PASS_ONE may lead to redundant cells returned" miss issue number 
This reverts commit c71da858ad.
 2017-06-13 20:27:28 +08:00
huzheng c71da858ad FilterList with MUST_PASS_ONE may lead to redundant cells returned 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-12 23:55:25 -07:00
Vincent 650ef5cf59 HBASE-18137 Replication gets stuck for empty WALs 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-10 11:29:51 -07:00
Ashu Pachauri 6e3da5a39a HBASE-18192: Replication drops recovered queues on region server shutdown 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-09 19:54:17 -07:00
Ashu Pachauri 961337aadc HBASE-18092: Removing a peer does not properly clean up the ReplicationSourceManager state and metrics 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-09 11:14:40 -07:00
Gary Helmling 6c4980161b HBASE-18141 Regionserver fails to shutdown when abort triggered during RPC call 2017-06-08 18:33:19 -07:00
Alex Araujo ba3a816663 HBASE-18184 Add hbase-hadoop2-compat jar as MapReduce dependency 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-07 14:26:08 -07:00
Phil Yang 381c89b5cc HBASE-15576 Scanning cursor to prevent blocking long time on ResultScanner.next() 2017-06-07 11:37:22 +08:00
Chia-Ping Tsai 9cb57ae35e HBASE-18145 The flush may cause the corrupt data for reading 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-06 18:03:02 -07:00
Andrew Purtell e0dbafd7cc HBASE-18132 Low replication should be checked in period in case of datanode rolling upgrade (Allan Yang) 2017-06-06 17:21:11 -07:00
huzheng 9c1efc9f9d HBASE-18066: Get with closest_row_before on hbase:meta can return empty Cell during region merge/split 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-06 17:13:16 -07:00
Enis Soztutar ea3075e7fd HBASE-15160 Put back HFile's HDFS op latency sampling code and add metrics for monitoring (Yu Li and Enis Soztutar) 2017-06-06 14:41:02 -07:00
Ashish Singhi 356d4e9187 HBASE-9393 Hbase does not closing a closed socket resulting in many CLOSE_WAIT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-06 12:59:19 -07:00
tedyu 39e8e2fb58 HBASE-18005 read replica: handle the case that region server hosting both primary replica and meta region is down (huaxiang sun) 2017-06-06 09:07:17 -07:00
Xiang Li 69d3e332f1 HBASE-18150: Do not call FSUtils.setVersion() and FSUtils.checkVersion() when using checkRootDir() to check hbase.wal.dir 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-06 05:44:29 -07:00
anoopsamjohn 8252fe4fbb HBASE-18030 Per Cell TTL tags may get duplicated with increments/Append causing tags length overflow. 2017-06-06 12:26:21 +05:30
Vincent 3cf4433260 HBASE-15995 Separate replication WAL reading from shipping 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-03 09:48:57 -07:00
Guanghao Zhang b66a478e73 HBASE-18111 Replication stuck when cluster connection is closed 
Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-01 15:18:04 -07:00
Chinmay Kulkarni 46ee6e0fcc HBASE 17959 Canary timeout should be configurable on a per-table basis 
For branch-1: Added support for configuring read/write timeouts on a per-table basis
when in region mode.
Added unit test for per-table timeout checks.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-31 17:58:09 -07:00
Phil Yang e670be690f HBASE-18122 Scanner id should include ServerName of region server 2017-05-31 14:01:58 +08:00
Andrew Purtell 140c559a3a HBASE-18027 HBaseInterClusterReplicationEndpoint should respect RPC limits when batching edits 2017-05-30 14:25:36 -07:00
zhangduo 1a37f3be82 HBASE-18042 Client Compatibility breaks between versions 1.2 and 1.3 2017-05-27 17:56:05 +08:00
huzheng d8c1e0e004 HBASE-16011 TableSnapshotScanner and TableSnapshotInputFormat can produce duplicate rows if split table. 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-05-26 07:31:05 -07:00
tedyu 109e0d5485 HBASE-18099 FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish 2017-05-25 07:56:13 -07:00
Vincent a3c3f1012d HBASE-18060 Backport to branch-1 HBASE-9774 HBase native metrics and metric collection for coprocessors 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-24 13:20:44 -07:00
Balazs Meszaros f2ba52ac45 HBASE-18096 Limit HFileUtil visibility and add missing annotations 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-05-24 16:34:27 +08:00
Stephen Yuan Jiang 50708d9524 HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good (Stephen Yuan Jiang) 2017-05-23 13:10:07 -07:00
Guanghao Zhang 4249a1f3d9 HBASE-18069 Fix flaky test TestReplicationAdminWithClusters#testDisableAndEnableReplication 2017-05-22 17:31:01 +08:00
zhangduo affd6ddf32 HBASE-18081 The way we process connection preamble in SimpleRpcServer is broken 2017-05-21 22:00:27 +08:00
tedyu c03f003f4f HBASE-18035 Meta replica does not give any primaryOperationTimeout to primary meta region (huaxiang sun) 2017-05-21 06:39:33 -07:00
huzheng f9dc4cad63 HBASE-11013 Clone Snapshots on Secure Cluster Should provide option to apply Retained User Permissions 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2017-05-18 17:54:53 +08:00
Andrew Purtell 292226690a HBASE-18043 Institute a hard limit for individual cell size that cannot be overridden by clients 2017-05-15 18:03:24 -07:00
Andrew Purtell 0a4528225c HBASE-18014 A case of Region remain unassigned when table enabled (Allan Yang) 2017-05-12 16:59:38 -07:00
Chia-Ping Tsai f81486445c HBASE-17887 Row-level consistency is broken for read 2017-05-12 19:44:16 +08:00
Andrew Purtell 9bf5bc1986 Amend HBASE-18000 Make sure we always return the scanner id with ScanResponse (Karan Mehta and Duo Zhang) 
Move getRegionScanner() call below builder.setScannerId() to handle a
corner case.
 2017-05-11 14:38:28 -07:00
Roman Nikitchenko ea89047abf HBASE-8758 Error in RegionCoprocessorHost class preScanner method documentation 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-05-10 11:34:18 +08:00
Andrew Purtell 0f6a2c4113 HBASE-17924 Consider sorting the row order when processing multi() ops before taking rowlocks 
(Allan Yang)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 2017-05-08 17:04:50 -07:00
Allan Yang 9a97e28bdf HBASE-17471 Region Seqid will be out of order in WAL if using mvccPreAssign 
Signed-off-by: Yu Li <liyu@apache.org>
 2017-05-06 23:29:49 +08:00
zhangduo 408645c4ef HBASE-18000 Make sure we always return the scanner id with ScanResponse 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-05 11:49:07 -07:00
tedyu dd321c4a35 HBASE-17991 Add more details about compaction queue on /dump (Guangxu Cheng) 2017-05-04 08:28:02 -07:00
Jingyun Tian a5f52360f6 HBASE-17993 Delete useless info log in RpcServer.processResponse 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-05-04 20:59:10 +08:00
Chinmay Kulkarni e08277ac8f Addendum patch for HBASE-12870. Added check for null pointer. 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-03 14:03:27 -07:00
Guanghao Zhang 9f25836d99 HBASE-17958 Avoid passing unexpected cell to ScanQueryMatcher when optimize SEEK to SKIP 2017-05-03 21:39:30 +08:00
ckulkarni d2fcfd74d9 HBASE-12870 "Major compaction triggered" and "Skipping major compaction" messages lack the region information 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-02 10:54:37 -07:00
Allan Yang 262c66f614 HBASE-17757 Unify blocksize after encoding to decrease memory fragment 
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
 2017-05-02 10:06:33 +05:30
QilinCao 9a71bacdf0 HBASE-17835 Spelling mistakes in the Java source 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-04-28 11:40:47 -05:00
Steen Manniche 0f5932b059 HBASE-17817 add table name to output (if available) when removing coprocessors 
Amending-Author: Sean Busbey <busbey@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-04-28 11:39:54 -05:00
huaxiang sun cdda1d0302 HBASE-17904 Get runs into NoSuchElementException when using Read Replica, with hbase. ipc.client.specificThreadForWriting to be true and hbase.rpc.client.impl to be org.apache.hadoop.hbase.ipc.RpcClientImpl 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-04-28 06:53:37 +08:00
Guanghao Zhang aca4eb8d5c HBASE-9899 for idempotent operation dups, return the result instead of throwing conflict exception (addendum for branch-1) 2017-04-27 17:08:21 +08:00
Karan Mehta 78c64c360f HBASE-17965 Canary tool should print the regionserver name on failure 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 2017-04-26 18:25:31 -07:00
zhangduo cbae657632 HBASE-17956 Raw scan should ignore TTL 2017-04-26 15:34:14 +08:00
tedyu 5f1691d06e HBASE-17302 The region flush request disappeared from flushQueue - addendum (Guangxu Cheng) 2017-04-22 15:09:29 -07:00
Abhishek Singh Chouhan d69a6366f6 HBASE-17937 Memstore size becomes negative in case of expensive postPut/Delete Coprocessor call 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-04-21 08:50:53 +08:00
zhangduo c8c2e07aa1 HBASE-17930 Avoid using Canary.sniff in HBaseTestingUtility 2017-04-18 18:17:38 +08:00
gjacoby a26de9b51e HBASE-17912 - Avoid major compactions on region server startup 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-04-17 19:43:03 -07:00
Xiang Li cb2f2a7d1b HBASE-17916 Error message not clear when the permission of staging dir 
is not as expected

Signed-off-by: Michael Stack <stack@apache.org>
 2017-04-14 08:12:09 -07:00
qgxiaozhan 1afb86a1c1 HBASE-17816 HRegion#mutateRowWithLocks should update writeRequestCount metric 
Signed-off-by: Jerry He <jerryjch@apache.org>
 2017-04-11 18:08:06 -07:00
Nemo Chen 4030facc99 HBASE-16469 Several log refactoring/improvement suggestions 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-04-11 14:28:47 -05:00
Tomu Tsuruhara a6e9de3a0e HBASE-17871 scan#setBatch(int) call leads wrong result of VerifyReplication 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-04-06 07:00:36 -07:00
Yi Liang 4057a6c89c HBASE-17861: Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-04-04 07:22:31 -07:00
Karan Mehta 19e4e4d49a HBASE-17698 ReplicationEndpoint choosing sinks 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-03-31 16:31:29 -07:00
Ramkrishna 589a0e2efb HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & Ram)) 2017-03-30 17:03:36 +05:30
tedyu d0139a8777 HBASE-17287 Master becomes a zombie if filesystem object closes 2017-03-29 04:50:49 -07:00
Ashu Pachauri babb857940 HBASE-16755 Honor flush policy under global memstore pressure 
Signed-off-by: Gary Helmling <garyh@apache.org>
 2017-03-27 11:48:17 -07:00
Chia-Ping Tsai 006bfc7054 HBASE-17623 Reuse the bytes array when building the hfile block 2017-03-25 23:51:14 +08:00
zhangduo 849ab5ff29 HBASE-17595 addendum fix the problem for mayHaveMoreCellsInRow 2017-03-23 20:35:12 +08:00
tedyu 9726c71681 HBASE-17798 RpcServer.Listener.Reader can abort due to CancelledKeyException (Guangxu Cheng) 2017-03-21 08:06:56 -07:00
zhangduo b973d3fd46 HBASE-17584 Expose ScanMetrics with ResultScanner rather than Scan 2017-03-20 16:32:20 +08:00
Chia-Ping Tsai 02d9bf0c57 HBASE-17803 PE always re-creates table when we specify the split policy 2017-03-19 19:10:24 +08:00
Michael Stack d1ea718e43 HBASE-17501 Revert "Revert "guard against NPE while reading FileTrailer and HFileBlock"" 
This reverts commit e92a147e19.

This is a revert of a revert -- i.e. a restore -- just so I can change
the commit message to add the JIRA number.
 2017-03-16 14:56:04 -07:00
Michael Stack e92a147e19 Revert "guard against NPE while reading FileTrailer and HFileBlock" 
This reverts commit c97905a962.

This is a revert just so I can add the JIRA number to the commit
message.
 2017-03-16 14:55:41 -07:00
zhangduo 094e9a311b HBASE-16584 Backport the new ipc implementation in HBASE-16432 to branch-1 2017-03-16 23:00:30 +08:00
tedyu 318298047b HBASE-17779 disable_table_replication returns misleading message and does not turn off replication (Janos Gub) 2017-03-14 12:34:04 -07:00
James Moore c97905a962 guard against NPE while reading FileTrailer and HFileBlock 
guard against NPE from FSInputStream#seek

Signed-off-by: Michael Stack <stack@apache.org>
 2017-03-13 14:53:57 -07:00
Guanghao Zhang 759d63b15c HBASE-17773 VerifyReplication tool wrongly emits Invalid arguments error (Tom Tsuruhara) 2017-03-13 17:53:43 +08:00
Guanghao Zhang 531598d67f HBASE-17746 TestSimpleRpcScheduler.testCoDelScheduling is broken 2017-03-13 11:10:33 +08:00
zhangduo dc84c7b9a7 HBASE-17712 Addendum modify log message 2017-03-10 16:01:45 +08:00
Stephen Yuan Jiang 7dfb3b3194 HBASE-17764 Solve TestMultiSlaveReplication flakiness (Stephen Yuan Jiang) 2017-03-09 18:38:57 -08:00
Ashu Pachauri 20cb390529 HBASE-15429 Add split policy for busy regions 
Signed-off-by: Gary Helmling <garyh@apache.org>
 2017-03-09 17:33:53 -08:00
Andrew Purtell 0bc9ad0628 HBASE-17731 Fractional latency reporting in MultiThreadedAction 2017-03-09 16:54:38 -08:00
Ashu Pachauri 598dd9bb0e HBASE-16977 VerifyReplication should log a printable representation of the row keys 
Signed-off-by: Gary Helmling <garyh@apache.org>
 2017-03-09 16:36:10 -08:00
Umesh Agashe 3d9520b140 HBASE-17761: Test TestRemoveRegionMetrics.testMoveRegion fails intermittently because of race condition 
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
 2017-03-09 14:26:10 -08:00
Esteban Gutierrez c70769ea89 HBASE-15941 HBCK repair should not unsplit healthy splitted region 
Signed-off-by: Michael Stack <stack@apache.org>
 2017-03-08 22:03:47 -08:00
Michael Stack ca5b8a44a4 HBASE-17718 Difference between RS's servername and its ephemeral node cause SSH stop working 
This patch reverts HBASE-9593 -- i.e. registering in zk before we
register with master putting it back to how it was where we register
in zk AFTER we report for duty with the master (because then we'll
register in zk with the name the master gave us). It then fixes the
problem reported in HBASE-9593 in an alternate fashion by checking
for a RS znode if we failed a connect on assign; if none found, we
remove a server from online servers list.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Make move method available to tests.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 Correct method name changing moveFromOnelineToDeadServers to
 moveFromOnlineToDeadServers

 Add actual fix which is call to checkForRSznode if exception trying to
 open a region; if none found, call expire on the server so it gets
 removed from the list of online servers.

 This patch exposes sloppyness in the waitForRegionServers around our
 current case where Master is hosting regions but ONLY hbase:meta;
 in this case we need to wait on at least another server to report
 in beyond Master (we weren't but stuff was 'working' because of the
 early registration of RS nodes in zk).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Make 'killed' available to tests.

 Put registry of ephemeral node back to where it was originally,
 so it is AFTER we get response from Master on registering for duty
 so we can put our znode up in zk with the name the Master gave us
 rather than local name (which could be unknown to the Master).
   private boolean stopping = false;

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenInitializing.java
 Cleanup and test of new cleanup.
 2017-03-08 10:47:53 -08:00
tedyu f34709e1b7 HBASE-17460 enable_table_replication can not perform cyclic replication of a table (Janos Gub) 2017-03-08 01:27:35 -08:00