Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,640 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

5335 Commits

Author SHA1 Message Date
Pankaj Kumar 343cdda9ae HBASE-19364, Truncate_preserve fails with table when replica region > 1 2018-02-23 14:13:02 -08:00
Sean Busbey ba7a936f74 Revert "HBASE-9465 Push entries to peer clusters serially" 
This reverts commit 441bc050b9.

 Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
	hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/WALProtos.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-02-23 14:42:15 +08:00
Sean Busbey 0a284d2b76 Revert "HBASE-17010 Serial replication should handle daughter regions being assigned to another RS (Phil Yang)" 
This reverts commit 97276da9a7.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-02-23 14:42:04 +08:00
tedyu 4d9589b165 HBASE-20052 TestRegionOpen#testNonExistentRegionReplica fails due to NPE 2018-02-22 12:41:13 -08:00
Michael Stack cadc2efd12 HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory; ADDEDNDUM to fix TestRegionOpen failure 2018-02-22 09:27:46 -08:00
Chia-Ping Tsai af1f7cf431 HBASE-20016 TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky 2018-02-22 09:44:37 +08:00
Pankaj Kumar a189674b4c HBASE-19553, Old replica regions should be cleared from AM memory after primary region split or merge 
Signed-off-by: Ted Yu <tedyu@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2018-02-21 16:52:40 -08:00
Esteban Gutierrez 9e7e58d551 HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory 2018-02-21 16:25:13 -08:00
Andrew Purtell c5e692c1dc HBASE-20027 Add test TestClusterPortAssignment 
LocalHBaseCluster forces random port assignment for sake of concurrent unit test
execution friendliness, but we still need a positive test for RPC and info port
assignment.
 2018-02-21 13:12:26 -08:00
Toshihiro Suzuki c0bca137ff HBASE-20008 [backport] NullPointerException when restoring a snapshot after splitting a region 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-02-20 20:29:08 -08:00
Chia-Ping Tsai 21f6830fe1 HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-02-20 16:59:44 -08:00
Michael Stack fd8189d31d Revert "HBASE-19970 Remove unused functions from TableAuthManager." 
This reverts commit e6ce789b6f.
 2018-02-20 11:12:36 -08:00
Michael Stack 071281cf52 Revert "HBASE-19970 (addendum for 1.x only) Remove unused functions from TableAuthManager." 
This reverts commit 0f79c497c5.
 2018-02-20 11:12:33 -08:00
chenheng 7990546efe HBASE-14897 TestTableLockManager.testReapAllTableLocks is flakey 
(cherry picked from commit 47082579a8)
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-02-20 08:50:22 -06:00
Ben Lau 28ebd29f0f HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of a KeeperException 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-02-14 17:23:23 -08:00
Thiruvel Thirumoolan e65004aeea HBASE-19996 Some nonce procs might not be cleaned up (follow up HBASE-19756) 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-02-14 09:17:13 -08:00
tedyu 335b8a8e14 HBASE-19979 ReplicationSyncUp tool may leak Zookeeper connection (Pankaj Kumar) 2018-02-13 08:11:03 -08:00
Chia-Ping Tsai f77b42b2af HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner 2018-02-13 21:29:01 +08:00
Apekshit Sharma 0f79c497c5 HBASE-19970 (addendum for 1.x only) Remove unused functions from TableAuthManager. 2018-02-12 16:31:09 -08:00
Apekshit Sharma e6ce789b6f HBASE-19970 Remove unused functions from TableAuthManager. 
Functions deleted: setTableUserPermissions, setTableGroupPermissions, setNamespaceUserPermissions,
setNamespaceGroupPermissions, writeTableToZooKeeper, writeNamespaceToZooKeeper
 2018-02-12 16:20:36 -08:00
huzheng 7ced28f099 HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink 2018-02-11 14:58:36 +08:00
Chia-Ping Tsai 0cbb19201c HBASE-19900 Region-level exception destroy the result of batch 2018-02-07 08:30:55 +08:00
Balazs Meszaros 5fa64fb72e HBASE-19886 Display maintenance mode in shell, web UI 2018-02-06 15:59:44 -08:00
Toshihiro Suzuki 92f733db4b HBASE-19934 HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-02-06 11:19:22 -08:00
Rahul Gidwani 673e809250
HBASE-19528 - Major Compaction Tool 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-02-06 05:39:25 -08:00
Ashish Singhi bdeab93196 HBASE-19905 ReplicationSyncUp tool will not exit if a peer replication is disabled 
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
 2018-02-04 18:24:32 +05:30
Andrew Purtell f35bcd2fe3 HBASE-19858 Backport HBASE-14061 (Support CF-level Storage Policy) to branch-1 
HBASE-14061 Support CF-level Storage Policy
HBASE-14061 Support CF-level Storage Policy (addendum)
HBASE-14061 Support CF-level Storage Policy (addendum2)
HBASE-15172 Support setting storage policy in bulkload
HBASE-17538 HDFS.setStoragePolicy() logs errors on local fs
HBASE-18015 Storage class aware block placement for procedure v2 WALs
HBASE-18017 Reduce frequency of setStoragePolicy failure warnings
Default storage policy if not configured cannot be "NONE"
HBASE-19016 Coordinate storage policy property name for table schema and bulkload
Fix checkstyle warnings
Addressed additional review feedback on backport
 2018-02-01 17:10:39 -08:00
binlijin e1bcf50b10 HBASE-19728 Add lock to filesCompacting in all place. 2018-01-31 17:01:58 +08:00
Thiruvel Thirumoolan 2c5140e151 HBASE-19756 Master NPE during completed failed proc eviction 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-01-24 16:48:54 -08:00
Rahul Gidwani 73ed127039 HBASE-15321 - Ability to open a HRegion from hdfs snapshot. 2018-01-22 12:22:57 -08:00
huaxiangsun 0f20c5e0fd HBASE-19163 "Maximum lock count exceeded" from region server's batch processing 
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-01-19 11:24:19 -08:00
Scott Wilson 779371625b HBASE-19816 Refresh repliation sinks on UnknownHostException 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-01-17 16:26:25 -08:00
Rahul Gidwani 48025cc84b HBASE-11409 - Add more flexibility for input directory structure to LoadIncrementalHFiles 2018-01-12 13:35:26 -08:00
Guangxu Cheng b29a138ecc HBASE-19483 Add proper privilege check for rsgroup commands addendum 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-01-11 11:00:07 -08:00
Chia-Ping Tsai 0ef6530aa1 HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout. 
Signed-off-by: Apekshit Sharma <appy@apache.org>
 2018-01-11 10:39:42 -08:00
Guangxu Cheng 6f1dd258b1 HBASE-19483 Add proper privilege check for rsgroup commands 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-01-10 02:25:47 -08:00
Toshihiro Suzuki e6453ab84b HBASE-19424 Fix NPE in "/metrics" servlet. 
Signed-off-by: Apekshit Sharma <appy@apache.org>
 2018-01-08 15:09:27 -08:00
BELUGA BEHR fa07c211ac HBASE-19684 BlockCacheKey toString Performance 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-01-08 17:06:44 +08:00
Andrew Purtell b528cd01b7 HBASE-19708 Avoid NPE when the RPC listener's accept channel is closed 2018-01-05 09:58:03 -08:00
Rajeshbabu Chintaguntla b5ec4d381c HBASE-15580 Tag coprocessor limitedprivate scope to StoreFile.Reader 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-01-04 14:15:03 +08:00
Michael Stack 939e016425
HBASE-19588 Additional jar dependencies needed for mapreduce 
PerformanceEvaluation

Signed-off-by: Albert Chu <chu11@llnl.gov>
 2018-01-03 21:33:25 -06:00
Chia-Ping Tsai 2c30c9bbb6 HBASE-19490 Rare failure in TestRateLimiter 2018-01-04 10:33:46 +08:00
Josh Elser 490728ae7f HBASE-19691 Removes Global(A) requirement for getClusterStatus 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-01-03 19:15:49 -05:00
huaxiang sun fd33b3589e HBASE-18625 Splitting of region with replica, doesn't update region list in serverHolding. A server crash leads to overlap. 
Signed-off-by: ramkrishna.s.vasudevan<ramkrishna.s.vasudevan@intel.com>
 2018-01-03 12:32:44 -08:00
Jingyun Tian 6faed49ad1 HBASE-19358 Improve the stability of splitting log when do fail over 
Signed-off-by: Yu Li <liyu@apache.org>
 2018-01-02 17:25:49 +08:00
tedyu e866e837fb HBASE-19551 hbck -boundaries doesn't work correctly (Toshihiro Suzuki) 2017-12-30 23:18:12 -08:00
Peter Somogyi 528eb10826 HBASE-19545 Replace getBytes(StandardCharsets.UTF_8) with Bytes.toBytes 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-12-27 20:15:01 +08:00
tedyu 64d2ebf9f3 HBASE-19514 TestJMXListener now uses random port 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2017-12-22 18:51:44 +01:00
ramkrish86 d28732fd98 HBASE-19468 FNFE during scans and flushes (Ram) 2017-12-20 17:06:27 +05:30
zhangduo 08e948b188 HBASE-19546 TestMasterReplication.testCyclicReplication2 uses wrong assertion 2017-12-18 19:59:32 +08:00
ramkrish86 ff023bcc4b HBASE-19485 Minor improvement to TestCompactedHFilesDischarger (Ram) 2017-12-14 13:19:55 -08:00
Chia-Ping Tsai 54ca1db6bd HBASE-19502 Make sure we have closed all StoreFileScanner if we fail to open any StoreFileScanners 2017-12-13 21:47:27 +08:00
Jan Hentschel b3167a0e50 Revert "HBASE-17425 Fixed deprecated calls in TestUpdateConfiguration" 
This reverts commit c434cab5d1.
 2017-12-11 22:56:33 +01:00
Jan Hentschel c434cab5d1 HBASE-17425 Fixed deprecated calls in TestUpdateConfiguration 2017-12-09 14:44:50 +01:00
Andrew Purtell bae3b0b966 HBASE-19465 Required httpcore and httpclient jars not included in binary distribution 2017-12-08 15:02:10 -08:00
Zach York 9e34d99c07 HBASE-19435 Reopen Files for ClosedChannelException in BucketCache 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-12-07 15:45:51 -08:00
Andrew Purtell 5398d83bc7 HBASE-19446 Misspelled 'default' in SimpleRpcScheduler 2017-12-06 17:11:15 -08:00
Andrew Purtell 70c7688271 HBASE-19447 INFO level logging of GetClusterStatus from HMaster is too chatty 2017-12-06 17:05:19 -08:00
Pankaj Kumar f88304d162 HBASE-19442 Backport HBASE-19065 to branch-1 (HRegion#bulkLoadHFiles() should wait for concurrent Region#flush() to finish). 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-12-06 14:18:44 -08:00
Andrew Purtell 6c2d51132f HBASE-19445 PerformanceEvaluation NPE processing split policy option 2017-12-06 14:10:11 -08:00
Andrew Purtell 1fe75f98d3 HBASE-19421 branch-1 does not compile against Hadoop 3.0.0 2017-12-04 15:48:38 -08:00
Guanghao Zhang 9d481b1c43 HBASE-19239 (addendum) Fix findbugs and error-prone warnings (branch-1) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-12-04 10:18:53 -08:00
tedyu cb3bef3574 HBASE-19056 TestCompactionInDeadRegionServer is top of the flakies charts 2017-12-04 06:35:41 -08:00
binlijin 15d16281a1 HBASE-19290 Reduce zk request when doing split log 2017-12-04 18:36:11 +08:00
huzheng 4ec59d018d HBASE-19252 Move the transform logic of FilterList into transformCell() method to avoid extra ref to question cell 2017-12-04 10:19:16 +08:00
Guanghao Zhang 25cc8feeb2 HBASE-19366 Backport to branch-1 HBASE-19035 Miss metrics when coprocessor use region scanner to read data 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-12-02 09:54:21 -08:00
Guanghao Zhang 525b2852e8 HBASE-19396 Fix flaky test TestHTableMultiplexerFlushCache 2017-12-02 19:43:32 +08:00
Andrew Purtell d2a54c98ba HBASE-19406 Fix CompactionRequest equals and hashCode 2017-12-01 19:15:05 -08:00
Andrew Purtell ab1d5e05a8 HBASE-19395 [branch-1] TestEndToEndSplitTransaction.testMasterOpsWhileSplitting fails with NPE 2017-12-01 15:31:52 -08:00
Guangxu Cheng 6938720c12 HBASE-19326 Remove decommissioned servers from rsgroup 
Signed-off-by: Michael Stack <stack@apache.org>

Amending-Author: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminClient.java
	hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
	hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java
	hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroups.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
	hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb
 2017-12-01 12:15:43 -08:00
Michael Stack 2f7a6f21eb HBASE-18233 We shouldn't wait for readlock in doMiniBatchMutation in case of deadlock (Allan Yang) 
This patch plus a sorting of the batch (HBASE-17924) fixes a regression
in Increment/CheckAndPut-style operations.

Signed-off-by: Yu Li <carp84@gmail.com>
Signed-off-by: Allan Yang <allanwin@163.com>
 2017-11-30 17:10:27 -08:00
Sergey Soldatov 143ceb97ba HBASE-19393 HTTP 413 FULL head while accessing HBase UI using SSL. 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-11-30 17:03:32 -08:00
Harshal Deepakkumar Jain a4b9ac6050 HBASE-19388 - Incorrect value is being set for Compaction Pressure in RegionLoadStats object inside HRegion class 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-11-30 15:07:45 -08:00
Josh Elser 4846300c8b HBASE-19285 Implements table-level latency histograms 
For a egionserver's view of a table (the regions
that belong to a table hosted on a regionserver),
this change tracks the latencies of operations that
affect the regions for this table.

Tracking at the per-table level avoids the memory bloat
and performance impact that accompanied the previous
per-region latency metrics while still providing important
details for operators to consume.

Signed-Off-By: Andrew Purtell <apurtell@apache.org>
 2017-11-30 17:45:23 -05:00
Michael Stack 1e00673047 HBASE-19385 [1.3] TestReplicator failed 1.3 nightly 2017-11-29 23:12:51 -08:00
Andrew Purtell 4c413e0c50 HBASE-19376 Fix more binary compatibility problems with branch-1.4 / branch-1 2017-11-29 14:07:36 -08:00
Peter Somogyi dce6285c21
HBASE-19188 Build fails on branch-1 using maven-3.5.2 
Changing tomcat:jasper-runtime scope to compile in
hbase-server, hbase-rest and hbase-thrift

Signed-off-by: Michael Stack <stack@apache.org>
 2017-11-29 12:28:09 -08:00
libisthanks 41b4877950
HBASE-18090 Improve TableSnapshotInputFormat to allow more multiple mappers per region 
Signed-off-by: Michael Stack <stack@apache.org>
 2017-11-28 14:56:38 -08:00
Chia-Ping Tsai 24d82195cb HBASE-19348 Fix error-prone errors for branch-1 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-11-27 18:29:11 -08:00
tedyu f7f425e53e HBASE-19300 TestMultithreadedTableMapper fails in branch-1.4 2017-11-27 11:41:46 -08:00
Guangxu Cheng de16690b7a HBASE-19325 Pass a list of server name to postClearDeadServers 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-11-28 02:39:55 +08:00
Gary Helmling 20d811121f HBASE-19332 DumpReplicationQueues misreports total WAL size 2017-11-22 16:54:13 -08:00
Andrew Purtell d80d3fa454 HBASE-19239 Fix findbugs and error-prone issues 
Fixes for hbase-server
 2017-11-17 17:12:36 -08:00
Michael Stack 1bde8656b1 HBASE-19276 RegionPlan should correctly implement equals and hashCode 
Andrew's patch that adds equals and hash but revamping compare also.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-11-16 14:06:46 -08:00
Michael Stack 641e797e02
HBASE-19245 MultiTableInputFormatBase#getSplits creates a Connection per Table 
We make one Connection only instead of a Connection per table (Change is
just moving one line it involves right-shifting body of the function)
 2017-11-14 21:48:45 -08:00
Lars Hofhansl f9833a7802 HBASE-12091 Optionally ignore edits for dropped tables for replication. 2017-11-14 16:33:12 -08:00
zhangduo c0639d271a HBASE-19243 Start mini cluster once before class for TestFIFOCompactionPolicy 2017-11-14 15:55:51 +08:00
Mike Drob dbcda15ae7 HBASE-19240 more error-prone results 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-11-13 18:23:10 -08:00
Andrew Purtell 45116c8a52 HBASE-19250 TestClientClusterStatus is flaky 2017-11-13 16:52:36 -08:00
Andrew Purtell 6961526573 HBASE-12350 Backport error-prone build support to branch-1 and branch-2 2017-11-09 15:34:09 -08:00
Mike Drob 027ef60a75 HBASE-19195 error-prone fixes for client, mr, and server 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-11-09 11:37:43 -08:00
Andrew Purtell c5e60d16b4 HBASE-19194 TestRSGroupsBase has some always false checks 2017-11-07 17:14:03 -08:00
huzheng 264cf0d473 HBASE-18993 Backport patches in HBASE-18410 to branch-1.x branches 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-11-07 14:00:27 -08:00
Michael Stack ca68d77862 HBASE-19198 TestIPv6NIOServerSocketChannel fails; unable to bind 2017-11-06 21:21:34 -08:00
Michael Stack a2ae58f6ee
HBASE-19102 TestZooKeeperMainServer fails with KeeperException 
Reapplication after fixing compile issue.
 2017-11-06 20:40:14 -08:00
Chia-Ping Tsai 56fd75e8e2 HBASE-19131 Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook 2017-11-05 09:55:10 +08:00
Michael Stack 9ad6e0422f
Revert "HBASE-19102 TestZooKeeperMainServer fails with KeeperException" 
This reverts commit 6547128789.
 2017-11-03 12:16:32 -07:00
Michael Stack 6547128789
HBASE-19102 TestZooKeeperMainServer fails with KeeperException 
Signed-off-by: Michael Stack <stack@apache.org>
 2017-11-03 11:49:18 -07:00
Josh Elser de087931ec HBASE-19156 Remove vestiges of an unused regions-per-server CLI argument 
Signed-off-by: Ted Yu <tedyu@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2017-11-02 15:05:44 -04:00
churro 61753bd133 HBASE-19150 TestSnapshotWithAcl is flaky 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-11-01 18:17:58 -07:00
Andrew Purtell 0ac5b33747 HBASE-19138 Rare failure in TestLruBlockCache 
Wait for the block count to drop to zero after awaiting shutdown of the executor pool
 2017-10-31 11:36:19 -07:00
Andrew Purtell 826e11a15d HBASE-19125 TestReplicator is flaky 2017-10-30 15:10:26 -07:00
Andrew Purtell 0d26ca7dfb HBASE-19113 Restore dropped constants from TableInputFormatBase for compatability 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-10-27 15:27:32 -07:00
tedyu e1a73b9144 HBASE-19072 Missing beak in catch block of InterruptedException in HRegion#waitForFlushes() 2017-10-23 19:38:45 -07:00
Andrew Purtell 64328caef0 HBASE-15631 Backport Regionserver Groups (HBASE-6721) to branch-1 (Francis Liu and Andrew Purtell) 2017-10-23 17:10:33 -07:00
tedyu 795f48c311 HBASE-19052 FixedFileTrailer should recognize CellComparatorImpl class in branch-1.x 2017-10-23 07:43:26 -07:00
Chia-Ping Tsai 11a4eb9af9 HBASE-18690 Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience 2017-10-23 10:38:06 +08:00
xiaowen147 ae6a951658 HBASE-19051 Add new split algorithm for num string 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-10-21 08:22:36 -07:00
tedyu 8d6902ae77 HBASE-18998 processor.getRowsToLock() always assumes there is some row being locked 2017-10-13 08:44:00 -07:00
zhangduo ff23e15769 HBASE-18552 Backport the server side change in HBASE-18489 to branch-1 2017-10-12 10:42:21 +08:00
tedyu 2828427219 HBASE-18959 Backport HBASE-18874 (HMaster abort message will be skipped if Throwable is passed null) to branch-1 (Pankaj Kumar) 2017-10-10 08:33:42 -07:00
Sean Busbey aa50971947 Revert "HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned" 
This reverts commit 256fc63007.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2017-10-09 17:06:14 -05:00
Sean Busbey c7dc0da849 Revert "HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT" 
This reverts commit ead1063244.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2017-10-09 17:06:09 -05:00
Peter Somogyi 8d77c1e954 HBASE-18957 add test that confirms 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will return results that match either of the FamilyFilters 
Amending-Author: Sean Busbey <busbey@apache.org>

Signed-off-by: Michael Stack <stack@apache.org>
 2017-10-09 17:05:39 -05:00
Maytee Chinavanichkit 50ef066c89 HBASE-18921 Fix Result.current ArrayIndexOutOfBoundsException 
Patch ArrayIndexOutOfBoundsException when current() is called after
advance() has already returned false

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-10-08 02:58:01 +08:00
Yi Liang fc783ef045 HBASE-16894 Create more than 1 split per region, generalize HBASE-12590 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-10-03 17:24:45 -07:00
libisthanks 1c7321f9d5 HBASE-18814 Make ScanMetrics enabled and add counter <HBase Counters, ROWS_SCANNED> into the MapReduce Job over snapshot 
Signed-off-by: Ashu Pachauri <ashu@apache.org>
 2017-10-02 18:25:52 -07:00
Pankaj Kumar b2943504b8 HBASE-18890, Backport HBASE-14499 (Master coprocessors shutdown will not happen on master abort) to branch-1 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-29 16:20:52 -07:00
Andrew Purtell 93ad1aba70 HBASE-18436 Add client-side hedged read metrics (Yun Zhao) 
Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicasClient.java
 2017-09-29 15:43:03 -07:00
shaofengshi aa86657a52 HBASE-18885 HFileOutputFormat2 hardcodes default FileOutputCommitter 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-27 06:58:43 -07:00
Andrew Purtell 3abc0458e9 Amend HBASE-18830 TestCanaryTool does not check Canary monitor's error code 
Adjust exception control flow to fix findbugs warning
NP_NULL_ON_SOME_PATH_EXCEPTION, Possible null pointer dereference of
regionSink in org.apache.hadoop.hbase.tool.Canary$RegionMonitor.run()
on exception path
 2017-09-26 10:38:09 -07:00
Michael Stack eeaa9aee66 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX (Biju Nair) 2017-09-25 20:51:11 -07:00
Chinmay Kulkarni 9d68535e8e HBASE-18830 TestCanaryTool does not check Canary monitor's error code 
Added assertion checks to make sure that the error code for the
ToolRunner run() method is used.

Testing Done: Checked that TestCanaryTool unit tests fail when there is
an error code in the current Canary run.

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 2017-09-25 19:17:22 -07:00
Chinmay Kulkarni 414e0ca292 HBASE-18762 Canary sink type cast error 
Changed the type hierarchy of Canary sinks to reduce confusion and avoid
cast errors.

Testing Done: Ran the TestCanaryTool.java test suite and confirmed that
the working is correct.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2017-09-25 19:17:22 -07:00
Andrew Purtell d74bdc3100 Amend HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas 
Remove now invalid unit test TestCorruptedRegionStoreFile
 2017-09-25 17:36:54 -07:00
Andrew Purtell 5a3d9cb225 HBASE-18786 FileNotFoundException should not be silently handled for primary region replicas 2017-09-22 12:41:03 -07:00
Zach York bafc6e523e HBASE-18478 Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible 
This provides significant cluster start time reduction for FileSystems which do not surface locality (S3).

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
 2017-09-19 16:30:43 -07:00
Abhishek Singh Chouhan 2845ddaf9e HBASE-18796 Admin#isTableAvailable returns incorrect result before daughter regions are opened 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-09-19 13:52:15 -07:00
tedyu ead1063244 HBASE-15410 Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT 2017-09-18 14:31:21 -07:00
andrewcheng 815673f7e4 HBASE-18131 Add an hbase shell command to clear deadserver list in ServerManager 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-16 09:33:08 -07:00
Biju Nair 7dc0a63c06 HBASE-18641 Include block content verification logic used in lruCache in bucketCache 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-16 06:12:43 -07:00
Andrew Purtell e5f80f36e2 Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4 
Disable units where argument matching is intermittently failing
 2017-09-15 10:42:04 -07:00
Andrew Purtell 469d6bf457 Revert "Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4" 
This reverts commit aa7c93c659.
 2017-09-15 10:42:04 -07:00
Reid Chan 862a717e37 HBASE-18801 Bulk load cleanup may falsely deem file deletion successful 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-15 02:15:06 -07:00
Andrew Purtell aa7c93c659 Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4 
Skip tests where mocking with custom matchers fails unexpectedly on
older JREs.
 2017-09-14 14:38:56 -07:00
Michael Stack 91422357f2 HBASE-18818 TestConnectionImplemenation fails 2017-09-14 12:31:01 -07:00
Andrew Purtell 0621486620 HBASE-18346 TestRSKilledWhenInitializing failing on branch-1 (branch-1.4) 
Temporarily disabled flaky test TestRSKilledWhenInitializing
 2017-09-13 14:48:36 -07:00
Andrew Purtell 3f785a83b2 Revert "HBASE-16488 Starting namespace and quota services in master startup asynchronizely (Stephen Yuan Jiang)" 
This reverts commit af359d03b5.
 2017-09-13 13:24:58 -07:00
Guangxu Cheng a8ac84f1a6 HBASE-18789 Displays the reporting interval of each RS on the Master page 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-13 09:39:09 -07:00
Abhishek Singh Chouhan 432ca7e3fb HBASE-18771 Incorrect StoreFileRefresh leading to split and compaction failures 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-09-12 18:08:50 -07:00
tedyu b6a12c843e HBASE-18652 Expose individual cache stats in a CombinedCache through JMX - revert due to test failure 2017-09-08 14:21:30 -07:00
tedyu ab8a667c45 HBASE-18641 Include block content verification logic used in lruCache in bucketCache - revert due to test failure 2017-09-08 12:45:47 -07:00
Biju Nair a6dbeba22a HBASE-18641 Include block content verification logic used in lruCache in bucketCache 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-08 12:07:02 -07:00
Biju Nair 5bfe1da984 HBASE-18652 Expose individual cache stats in a CombinedCache through JMX 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-06 13:30:44 -07:00
Biju Nair 5de4a7d898 HBASE-18737 Display configured max size of cache on RS UI 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-06 11:06:27 -07:00
Zach York 4733aa4655 HBASE-18757 Fix improper bitwise & in bucketcache offset calculation 
This correctly casts the operand to a long to avoid negative offsets created by sign extending the integer operand.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-09-05 15:10:08 -07:00
Chun-Hao Tang 8eacd32beb HBASE-18746 Throw exception with job.getStatus().getFailureInfo() when ExportSnapshot fails 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-09-05 02:04:09 +08:00
tedyu d5033fd350 HBASE-18743 HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default table are deleted (wenbang) 2017-09-04 07:52:57 -07:00
Cesar Delgado 8178305d4e HBASE-18675 Making {max,min}SessionTimeout configurable for MiniZooKeeperCluster 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-08-30 14:32:37 -07:00
Sean Mackrory aec4bf6bae HBASE-15947 Classes used only for tests included in main code base 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-08-30 10:53:31 +08:00
Vikas Vishwakarma f071e56a9e HBASE-18633 Add more info to understand the source/scenario of large batch requests exceeding threshold 2017-08-28 15:30:40 +05:30
Ajay Jadhav 7d605fe9c2 HBASE-18448 Added refresh HFiles coprocessor endpoint 
Signed-off-by: Michael Stack <stack@apache.org>
 2017-08-24 16:05:04 -07:00
Mike Drob 362a2924d2 HBASE-18628 Fix event pre-emption in ZKPermWatcher 
Instead of using an Atomic Reference to data and aborting when we detect
that new data comes in, use the native cancellation/pre-emption features
of Java Future.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-08-23 16:42:33 -07:00
Biju Nair 19a80c8234 HBASE-18532 Improve cache related stats rendered on RS UI 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-23 13:26:10 -07:00
Biju Nair ad22437d05 HBASE-18614 Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-23 10:51:27 -07:00
Michael Stack 338232f81f Revert "HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable" 
Lots of tests timing out. Revert for now.

This reverts commit 7a96682643.
 2017-08-22 08:31:55 -07:00
Michael Stack b8857b0a7b HBASE-16615 Fix flaky TestScannerHeartbeatMessages (Duo Zhang) 2017-08-21 16:01:26 -07:00
Andrew Purtell 19fffacd3a HBASE-18623 Frequent failed to parse at EOF warnings from WALEntryStream 2017-08-21 09:29:20 -07:00
Pankaj Kumar 7a96682643 HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable 
Signed-off-by: Michael Stack <stack@apache.org>
 2017-08-18 14:23:59 -07:00
Chia-Ping Tsai f61f09cdf2 HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to next column 2017-08-19 02:14:37 +08:00
Chia-Ping Tsai 10d65ee37f HBASE-18572 Delete can't remove the cells which have no visibility label 2017-08-19 01:56:29 +08:00
tedyu 64bef67558 HBASE-18617 FuzzyRowKeyFilter should not modify the filter pairs (vinisha) 2017-08-18 10:02:06 -07:00
Pankaj Kumar deeda60c69 HBASE-17617 Backport HBASE-16731 (Inconsistent results from the Get/Scan if we use the empty FilterList) to branch-1 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-08-17 18:31:57 +08:00
Zach York 1f1ab8c873 HBASE-18587 Fix flaky TestFileIOEngine 
This short circuits reads and writes with 0 length and also removes flakiness in TestFileIOEngine

Signed-off-by: Michael Stack <stack@apache.org>
 2017-08-16 14:50:11 -07:00
Andrew Purtell 3feb87b005 HBASE-18431 Mitigate compatibility concerns between branch-1.3 and branch-1.4 
- Refactor TableName into its own proto module and fix up users

- Move SnapshotDescription from Snapshot.proto back to HBase.proto

- Restore FastLongHistogram and TestFastLongHistogram;
  deprecate FastLongHistogram

- Move DeleteQueryTracker back to o.a.h.h.regionserver
 2017-08-16 11:34:18 -07:00
Ramkrishna 6255dc7001 HBASE-18251 Remove unnecessary traversing to the first and last keys in 
the CellSet (Toshihoro Suzuki)
 2017-08-16 10:50:58 -07:00
Ashish Singhi a49d43bfbf HBASE-18437 Revoke access permissions of a user from a table does not work as expected 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 2017-08-15 18:59:21 -07:00
Pankaj Kumar d7c6a0bf43 HBASE-18512, Region Server will abort with IllegalStateException if HDFS umask has limited scope 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-15 13:26:28 -07:00
tedyu 3ed765257e HBASE-18526 FIFOCompactionPolicy pre-check uses wrong scope (Vladimir Rodionov) 2017-08-15 11:09:55 -07:00
Zach York d6a781cf08 HBASE-18533 Expose BucketCache values to be configured 
Before this commit, BucketCache always used the default values.
This commit adds the ability to configure these values.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-14 13:28:42 -07:00
Jerry He 9078a034c4 HBASE-18522 Add RowMutations support to Batch 2017-08-14 09:21:53 -07:00
Esteban Gutierrez 578e29f96b HBASE-18025 CatalogJanitor should collect outdated RegionStates from the AM 2017-08-11 14:39:16 -05:00
Jan Hentschel b181f172ef HBASE-18197 Avoided to call job.waitForCompletion(true) two times 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-08-12 02:47:25 +08:00
Yu Li a03c2036e9 HBASE-18469 Correct RegionServer metric of totalRequestCount 2017-08-11 14:53:23 +08:00
Esteban Gutierrez 15bad3c036 HBASE-18024 HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists 2017-08-10 18:54:01 -05:00
Ashu Pachauri cab492d34f HBASE-18398: Snapshot operation fails with FileNotFoundException 2017-08-10 14:49:59 -07:00
Andrew Purtell 685ab19064 HBASE-18248 Warn if monitored RPC task has been tied up beyond a configurable threshold 2017-08-09 18:17:21 -07:00
Zach York 036fce3cf8 HBASE-18520 Add jmx value to determine true Master Start time; amendment 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-08 15:56:56 +08:00
Michael Stack b5a24b6696 Revert "HBASE-18520 Add jmx value to determine true Master Start time" 
This reverts commit 1ba949e030.
 2017-08-08 15:43:58 +08:00
Zach York 3b49e60e17 HBASE-18520 Add jmx value to determine true Master Start time 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-08 15:27:19 +08:00
Michael Stack ce91ab4871 Revert "HBASE-18520 Add jmx value to determine true Master Start time" 
This reverts commit 1ba949e030.
 2017-08-08 15:25:07 +08:00
Zach York 1ba949e030 HBASE-18520 Add jmx value to determine true Master Start time 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-08 15:02:58 +08:00
Zach York cad3d55892 HBASE-18520 Add jmx value to determine true Master Start time 
This is to determine how long it took in total for the master to start and finish initializing.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-08-05 22:34:26 -07:00
Chia-Ping Tsai a35dc0dddf HBASE-18480 The cost of BaseLoadBalancer.cluster is changed even if the rollback is done 2017-08-02 08:41:53 +08:00
James Taylor 80b85a857c HBASE-18487 Minor fixes in row lock implementation 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-08-01 11:28:28 -07:00
Abhishek Singh Chouhan abb9d88dce HBASE-18374 RegionServer Metrics improvements 2017-07-31 12:47:11 +05:30
Guanghao Zhang 59cad7d3e3 HBASE-18481 The autoFlush flag was not used in PE tool 2017-07-31 11:12:33 +08:00
Abhishek Singh Chouhan 9278037108 HBASE-15134 Add visibility into Flush and Compaction queues 2017-07-28 13:21:04 +05:30
Esteban Gutierrez e662d04321 HBASE-18362 hbck should not report split replica parent region from meta as errors (Huaxiang Sun) 
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
 2017-07-27 15:59:07 -05:00
Mike Drob aca8a124c0 HBASE-17922 Clean TestRegionServerHostname for hadoop3. 
Change-Id: I6f1514b1bc301be553912539e6a4192c2ccc782b
Signed-off-by: Apekshit Sharma <appy@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerHostname.java
 2017-07-26 15:27:58 -07:00
Andrew Purtell a4e1af1fcc HBASE-18054 log when we add/remove failed servers in client (Ali) 
Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/FailedServers.java
 2017-07-25 18:53:01 -07:00
huaxiang sun 9427d6c73c HBASE-18247 Hbck to fix the case that replica region shows as key in the meta table 
Change-Id: I2169290ec5a97a63747bc93ee37f36729c3ac916
 2017-07-25 17:08:53 -07:00
Josh Elser c7e1d67df7 HBASE-18023 Update row threshold warning from 1k to 5k (addendum) 2017-07-25 18:39:44 -04:00
rgidwani 26247996d2 HBASE-15816 Provide client with ability to set priority on Operations 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-07-21 17:12:25 -07:00
anoopsamjohn 9036556a33 HBASE-16993 BucketCache throw java.io.IOException: Invalid HFile block magic when configuring hbase.bucketcache.bucket.sizes. 2017-07-20 23:00:48 +05:30
Andrew Purtell cfd5b6b59f HBASE-18330 NPE in ReplicationZKLockCleanerChore 2017-07-19 15:46:45 -07:00
Chia-Ping Tsai 2da5b432a1 HBASE-18308 Eliminate the findbugs warnings for hbase-server 2017-07-20 00:36:16 +08:00
Phil Yang 980209579b HBASE-18390 Sleep too long when finding region location failed 2017-07-19 12:05:21 +08:00
Stephen Yuan Jiang af359d03b5 HBASE-16488 Starting namespace and quota services in master startup asynchronizely (Stephen Yuan Jiang) 2017-07-18 06:58:29 -07:00
tedyu 5a28437a74 HBASE-18377 Error handling for FileNotFoundException should consider RemoteException in openReader() 2017-07-18 06:50:02 -07:00
tedyu e1bf966f93 HBASE-18358 Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish' 2017-07-11 18:12:15 -07:00
Chia-Ping Tsai bec34ae432 HBASE-18295 The result contains the cells across different rows 2017-07-12 02:46:13 +08:00
Chia-Ping Tsai 44651e52d8 HBASE-18267 The result from the postAppend is ignored 2017-07-11 10:31:21 +08:00
tedyu 5d710bbe7f HBASE-18312 Ineffective handling of FileNotFoundException in FileLink.tryOpen() 2017-07-05 21:02:28 -07:00
Andrew Purtell 3dd55fa0c0 Set versions on branch-1 to 1.5.0-SNAPSHOT 2017-07-03 18:01:15 -07:00
Phil Yang 3381c6c453 HBASE-17931 Assign system tables to servers with highest version 2017-07-03 11:35:31 +08:00
David Harju 743f454d25 HBASE-18023 Log multi-* requests for more than threshold number of rows 
Signed-off-by: Josh Elser <elserj@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2017-06-30 18:45:36 -07:00
Sean Busbey 9264e8fb17 HBASE-17995 improve log messages during snapshot tests. 
Signed-off-by: Michael Stack <stack@apache.org>

 Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
 2017-06-30 10:16:05 -05:00
zhangduo 67b61e6d5a HBASE-16585 Rewrite the delegation token tests with Parameterized pattern 2017-06-30 16:26:34 +08:00
Kahlil Oppenheimer bb181ff50b HBASE-18164 Fast locality computation in balancer - addendum handles NaN 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-06-27 13:33:48 -05:00
Sean Busbey 386a6e87fe Revert "HBASE-18164 Fast locality computation in balancer - addendum handles NaN" 
This reverts commit 74e9d31474.

early commit missed some review feedback.
 2017-06-27 13:33:23 -05:00
Kahlil Oppenheimer 74e9d31474 HBASE-18164 Fast locality computation in balancer - addendum handles NaN 
-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.

-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer

-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-26 12:40:30 -07:00
Pankaj Kumar d5bba50794 HBASE-18167 OfflineMetaRepair tool may cause HMaster to abort always 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-22 02:11:07 -07:00
tedyu 3d9c544428 HBASE-18167 OfflineMetaRepair tool may cause HMaster to abort always - revert wrong version of patch 2017-06-22 02:10:25 -07:00
tedyu 940f4107b0 HBASE-18226 Disable reverse DNS lookup at HMaster and use the hostname provided by RegionServer (Duo Xu) 2017-06-21 18:23:30 -07:00
Stephen Yuan Jiang 105c5c36e6 HBASE-15691 ConcurrentModificationException in BucketAllocator 2017-06-21 15:23:59 -07:00
QilinCao 7b977695aa HBASE-18252 Resolve BaseLoadBalancer bad practice warnings 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-21 10:13:18 -07:00
Pankaj Kumar 01027f805b HBASE-18167 OfflineMetaRepair tool may cause HMaster to abort always 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-21 08:22:30 -07:00
Stephen Yuan Jiang 532e0dda16 HBASE-18036 Data locality is not maintained after cluster restart or SSH (Stephen Yuan Jiang) 2017-06-20 13:07:19 -07:00
Ashish Singhi dead08d668 HBASE-18212 reduce log level for unbuffer warning. 
In Standalone mode with local filesystem HBase logs Warning message:Failed to invoke 'unbuffer' method in class org.apache.hadoop.fs.FSDataInputStream

Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-06-20 02:18:44 -05:00
Kahlil Oppenheimer 4a2f475ed3 HBASE-18164 Fast locality computation in balancer 
-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.

-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer

-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
 2017-06-20 02:18:43 -05:00
Pankaj Kumar d5749bf8ed HBASE-18180 Possible connection leak while closing BufferedMutator in TableOutputFormat 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-19 06:39:27 -07:00
Andrew Purtell 316e02e3d8 HBASE-18219 Fix typo in constant HConstants.HBASE_CLIENT_MEAT_REPLICA_SCAN_TIMEOUT 2017-06-14 16:03:54 -07:00
huzheng 256fc63007 HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-06-13 20:28:04 +08:00
zhangduo d6e85b0511 Revert "FilterList with MUST_PASS_ONE may lead to redundant cells returned" miss issue number 
This reverts commit c71da858ad.
 2017-06-13 20:27:28 +08:00
huzheng c71da858ad FilterList with MUST_PASS_ONE may lead to redundant cells returned 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-12 23:55:25 -07:00
Vincent 650ef5cf59 HBASE-18137 Replication gets stuck for empty WALs 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-10 11:29:51 -07:00
Ashu Pachauri 6e3da5a39a HBASE-18192: Replication drops recovered queues on region server shutdown 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-09 19:54:17 -07:00
Ashu Pachauri 961337aadc HBASE-18092: Removing a peer does not properly clean up the ReplicationSourceManager state and metrics 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-09 11:14:40 -07:00
Gary Helmling 6c4980161b HBASE-18141 Regionserver fails to shutdown when abort triggered during RPC call 2017-06-08 18:33:19 -07:00
Alex Araujo ba3a816663 HBASE-18184 Add hbase-hadoop2-compat jar as MapReduce dependency 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-07 14:26:08 -07:00
Phil Yang 381c89b5cc HBASE-15576 Scanning cursor to prevent blocking long time on ResultScanner.next() 2017-06-07 11:37:22 +08:00
Chia-Ping Tsai 9cb57ae35e HBASE-18145 The flush may cause the corrupt data for reading 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-06 18:03:02 -07:00
Andrew Purtell e0dbafd7cc HBASE-18132 Low replication should be checked in period in case of datanode rolling upgrade (Allan Yang) 2017-06-06 17:21:11 -07:00
huzheng 9c1efc9f9d HBASE-18066: Get with closest_row_before on hbase:meta can return empty Cell during region merge/split 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-06 17:13:16 -07:00
Enis Soztutar ea3075e7fd HBASE-15160 Put back HFile's HDFS op latency sampling code and add metrics for monitoring (Yu Li and Enis Soztutar) 2017-06-06 14:41:02 -07:00
Ashish Singhi 356d4e9187 HBASE-9393 Hbase does not closing a closed socket resulting in many CLOSE_WAIT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-06 12:59:19 -07:00
tedyu 39e8e2fb58 HBASE-18005 read replica: handle the case that region server hosting both primary replica and meta region is down (huaxiang sun) 2017-06-06 09:07:17 -07:00
Xiang Li 69d3e332f1 HBASE-18150: Do not call FSUtils.setVersion() and FSUtils.checkVersion() when using checkRootDir() to check hbase.wal.dir 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-06 05:44:29 -07:00
anoopsamjohn 8252fe4fbb HBASE-18030 Per Cell TTL tags may get duplicated with increments/Append causing tags length overflow. 2017-06-06 12:26:21 +05:30
Vincent 3cf4433260 HBASE-15995 Separate replication WAL reading from shipping 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-06-03 09:48:57 -07:00
Guanghao Zhang b66a478e73 HBASE-18111 Replication stuck when cluster connection is closed 
Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HBaseInterClusterReplicationEndpoint.java

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-06-01 15:18:04 -07:00
Chinmay Kulkarni 46ee6e0fcc HBASE 17959 Canary timeout should be configurable on a per-table basis 
For branch-1: Added support for configuring read/write timeouts on a per-table basis
when in region mode.
Added unit test for per-table timeout checks.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-31 17:58:09 -07:00
Phil Yang e670be690f HBASE-18122 Scanner id should include ServerName of region server 2017-05-31 14:01:58 +08:00
Andrew Purtell 140c559a3a HBASE-18027 HBaseInterClusterReplicationEndpoint should respect RPC limits when batching edits 2017-05-30 14:25:36 -07:00
zhangduo 1a37f3be82 HBASE-18042 Client Compatibility breaks between versions 1.2 and 1.3 2017-05-27 17:56:05 +08:00
huzheng d8c1e0e004 HBASE-16011 TableSnapshotScanner and TableSnapshotInputFormat can produce duplicate rows if split table. 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-05-26 07:31:05 -07:00
tedyu 109e0d5485 HBASE-18099 FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish 2017-05-25 07:56:13 -07:00
Vincent a3c3f1012d HBASE-18060 Backport to branch-1 HBASE-9774 HBase native metrics and metric collection for coprocessors 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-24 13:20:44 -07:00
Balazs Meszaros f2ba52ac45 HBASE-18096 Limit HFileUtil visibility and add missing annotations 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-05-24 16:34:27 +08:00
Stephen Yuan Jiang 50708d9524 HBASE-18093 Overloading the meaning of 'enabled' in Quota Manager to indicate either quota disabled or quota manager not ready is not good (Stephen Yuan Jiang) 2017-05-23 13:10:07 -07:00
Guanghao Zhang 4249a1f3d9 HBASE-18069 Fix flaky test TestReplicationAdminWithClusters#testDisableAndEnableReplication 2017-05-22 17:31:01 +08:00
zhangduo affd6ddf32 HBASE-18081 The way we process connection preamble in SimpleRpcServer is broken 2017-05-21 22:00:27 +08:00
tedyu c03f003f4f HBASE-18035 Meta replica does not give any primaryOperationTimeout to primary meta region (huaxiang sun) 2017-05-21 06:39:33 -07:00
huzheng f9dc4cad63 HBASE-11013 Clone Snapshots on Secure Cluster Should provide option to apply Retained User Permissions 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2017-05-18 17:54:53 +08:00
Andrew Purtell 292226690a HBASE-18043 Institute a hard limit for individual cell size that cannot be overridden by clients 2017-05-15 18:03:24 -07:00
Andrew Purtell 0a4528225c HBASE-18014 A case of Region remain unassigned when table enabled (Allan Yang) 2017-05-12 16:59:38 -07:00
Chia-Ping Tsai f81486445c HBASE-17887 Row-level consistency is broken for read 2017-05-12 19:44:16 +08:00
Andrew Purtell 9bf5bc1986 Amend HBASE-18000 Make sure we always return the scanner id with ScanResponse (Karan Mehta and Duo Zhang) 
Move getRegionScanner() call below builder.setScannerId() to handle a
corner case.
 2017-05-11 14:38:28 -07:00
Roman Nikitchenko ea89047abf HBASE-8758 Error in RegionCoprocessorHost class preScanner method documentation 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2017-05-10 11:34:18 +08:00
Andrew Purtell 0f6a2c4113 HBASE-17924 Consider sorting the row order when processing multi() ops before taking rowlocks 
(Allan Yang)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 2017-05-08 17:04:50 -07:00
Allan Yang 9a97e28bdf HBASE-17471 Region Seqid will be out of order in WAL if using mvccPreAssign 
Signed-off-by: Yu Li <liyu@apache.org>
 2017-05-06 23:29:49 +08:00
zhangduo 408645c4ef HBASE-18000 Make sure we always return the scanner id with ScanResponse 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-05 11:49:07 -07:00
tedyu dd321c4a35 HBASE-17991 Add more details about compaction queue on /dump (Guangxu Cheng) 2017-05-04 08:28:02 -07:00
Jingyun Tian a5f52360f6 HBASE-17993 Delete useless info log in RpcServer.processResponse 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-05-04 20:59:10 +08:00
Chinmay Kulkarni e08277ac8f Addendum patch for HBASE-12870. Added check for null pointer. 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-03 14:03:27 -07:00
Guanghao Zhang 9f25836d99 HBASE-17958 Avoid passing unexpected cell to ScanQueryMatcher when optimize SEEK to SKIP 2017-05-03 21:39:30 +08:00
ckulkarni d2fcfd74d9 HBASE-12870 "Major compaction triggered" and "Skipping major compaction" messages lack the region information 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-05-02 10:54:37 -07:00
Allan Yang 262c66f614 HBASE-17757 Unify blocksize after encoding to decrease memory fragment 
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
 2017-05-02 10:06:33 +05:30
QilinCao 9a71bacdf0 HBASE-17835 Spelling mistakes in the Java source 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-04-28 11:40:47 -05:00
Steen Manniche 0f5932b059 HBASE-17817 add table name to output (if available) when removing coprocessors 
Amending-Author: Sean Busbey <busbey@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-04-28 11:39:54 -05:00
huaxiang sun cdda1d0302 HBASE-17904 Get runs into NoSuchElementException when using Read Replica, with hbase. ipc.client.specificThreadForWriting to be true and hbase.rpc.client.impl to be org.apache.hadoop.hbase.ipc.RpcClientImpl 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-04-28 06:53:37 +08:00
Guanghao Zhang aca4eb8d5c HBASE-9899 for idempotent operation dups, return the result instead of throwing conflict exception (addendum for branch-1) 2017-04-27 17:08:21 +08:00
Karan Mehta 78c64c360f HBASE-17965 Canary tool should print the regionserver name on failure 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 2017-04-26 18:25:31 -07:00
zhangduo cbae657632 HBASE-17956 Raw scan should ignore TTL 2017-04-26 15:34:14 +08:00
tedyu 5f1691d06e HBASE-17302 The region flush request disappeared from flushQueue - addendum (Guangxu Cheng) 2017-04-22 15:09:29 -07:00
Abhishek Singh Chouhan d69a6366f6 HBASE-17937 Memstore size becomes negative in case of expensive postPut/Delete Coprocessor call 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2017-04-21 08:50:53 +08:00
zhangduo c8c2e07aa1 HBASE-17930 Avoid using Canary.sniff in HBaseTestingUtility 2017-04-18 18:17:38 +08:00
gjacoby a26de9b51e HBASE-17912 - Avoid major compactions on region server startup 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-04-17 19:43:03 -07:00
Xiang Li cb2f2a7d1b HBASE-17916 Error message not clear when the permission of staging dir 
is not as expected

Signed-off-by: Michael Stack <stack@apache.org>
 2017-04-14 08:12:09 -07:00
qgxiaozhan 1afb86a1c1 HBASE-17816 HRegion#mutateRowWithLocks should update writeRequestCount metric 
Signed-off-by: Jerry He <jerryjch@apache.org>
 2017-04-11 18:08:06 -07:00
Nemo Chen 4030facc99 HBASE-16469 Several log refactoring/improvement suggestions 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2017-04-11 14:28:47 -05:00
Tomu Tsuruhara a6e9de3a0e HBASE-17871 scan#setBatch(int) call leads wrong result of VerifyReplication 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-04-06 07:00:36 -07:00
Yi Liang 4057a6c89c HBASE-17861: Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2017-04-04 07:22:31 -07:00
Karan Mehta 19e4e4d49a HBASE-17698 ReplicationEndpoint choosing sinks 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2017-03-31 16:31:29 -07:00
Ramkrishna 589a0e2efb HBASE-17837 Backport HBASE-15314 to branch-1.3 (Chunhui shen & Ram)) 2017-03-30 17:03:36 +05:30
tedyu d0139a8777 HBASE-17287 Master becomes a zombie if filesystem object closes 2017-03-29 04:50:49 -07:00
Ashu Pachauri babb857940 HBASE-16755 Honor flush policy under global memstore pressure 
Signed-off-by: Gary Helmling <garyh@apache.org>
 2017-03-27 11:48:17 -07:00
Chia-Ping Tsai 006bfc7054 HBASE-17623 Reuse the bytes array when building the hfile block 2017-03-25 23:51:14 +08:00
zhangduo 849ab5ff29 HBASE-17595 addendum fix the problem for mayHaveMoreCellsInRow 2017-03-23 20:35:12 +08:00
tedyu 9726c71681 HBASE-17798 RpcServer.Listener.Reader can abort due to CancelledKeyException (Guangxu Cheng) 2017-03-21 08:06:56 -07:00
zhangduo b973d3fd46 HBASE-17584 Expose ScanMetrics with ResultScanner rather than Scan 2017-03-20 16:32:20 +08:00
Chia-Ping Tsai 02d9bf0c57 HBASE-17803 PE always re-creates table when we specify the split policy 2017-03-19 19:10:24 +08:00
Michael Stack d1ea718e43 HBASE-17501 Revert "Revert "guard against NPE while reading FileTrailer and HFileBlock"" 
This reverts commit e92a147e19.

This is a revert of a revert -- i.e. a restore -- just so I can change
the commit message to add the JIRA number.
 2017-03-16 14:56:04 -07:00
Michael Stack e92a147e19 Revert "guard against NPE while reading FileTrailer and HFileBlock" 
This reverts commit c97905a962.

This is a revert just so I can add the JIRA number to the commit
message.
 2017-03-16 14:55:41 -07:00
zhangduo 094e9a311b HBASE-16584 Backport the new ipc implementation in HBASE-16432 to branch-1 2017-03-16 23:00:30 +08:00
tedyu 318298047b HBASE-17779 disable_table_replication returns misleading message and does not turn off replication (Janos Gub) 2017-03-14 12:34:04 -07:00
James Moore c97905a962 guard against NPE while reading FileTrailer and HFileBlock 
guard against NPE from FSInputStream#seek

Signed-off-by: Michael Stack <stack@apache.org>
 2017-03-13 14:53:57 -07:00
Guanghao Zhang 759d63b15c HBASE-17773 VerifyReplication tool wrongly emits Invalid arguments error (Tom Tsuruhara) 2017-03-13 17:53:43 +08:00
Guanghao Zhang 531598d67f HBASE-17746 TestSimpleRpcScheduler.testCoDelScheduling is broken 2017-03-13 11:10:33 +08:00