Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,727 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

7779 Commits

Author SHA1 Message Date
Wellington Chevreuil 02fd482630 HBASE-22200 - WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir 
Signed-off-by: Zach York <zyork@apache.org>
 2019-04-22 15:04:08 -07:00
zhangduo 8973885cf4 HBASE-22279 Add a getRegionLocator method in Table/AsyncTable interface 2019-04-22 18:39:53 +08:00
meiyi c015c6cac0 HBASE-22208 Create access checker and expose it in RS 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-22 08:14:50 +08:00
zhangduo 393c5b9502 HBASE-22261 Make use of ClusterStatusListener for async client 2019-04-21 22:26:58 +08:00
zhangduo 65fb40d7ee HBASE-22267 Implement client push back for async client 2019-04-21 16:51:55 +08:00
Bahram Chehrazy 22fca940be HBASE-22150 rssStub in HRegionServer is not thread safe and should not directly be used 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2019-04-17 11:28:39 -07:00
Ben Manes cadf321d53
HBASE-15560 W-TinyLFU based BlockCache 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-04-17 11:10:55 -07:00
Andrew Purtell f3532e4c26
HBASE-22235 OperationStatus.{SUCCESS|FAILURE|NOT_RUN} are not visible to 3rd party coprocessors 2019-04-16 13:19:56 -07:00
zhanggangxue 9a0daa8cbd HBASE-21257 misspelled words.[occured -> occurred] 2019-04-14 21:36:24 +08:00
Duo Zhang df255f639d HBASE-22207 Fix flakey TestAssignmentManager.testAssignSocketTimeout 2019-04-14 16:53:34 +08:00
Guanghao Zhang 69e3a4eb32 HBASE-22193 Add backoff when region failed open too many times 2019-04-13 11:14:34 +08:00
Jan Hentschel b5b1f86dc7
HBASE-22187 Removed remaining usages of ClusterConnection.clearRegionCache 2019-04-12 09:59:52 +02:00
Duo Zhang 33b66a3981 HBASE-22202 Fix new findbugs issues after we upgrade hbase-thirdparty dependencies 2019-04-12 14:39:25 +08:00
meiyi 61bc0b4741 HBASE-22084 Rename AccessControlLists to PermissionStorage 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-12 14:22:57 +08:00
Josh Elser 0f155b9ec9 HBASE-22144 Correct MultiRowRangeFilter to work with reverse scans 
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
 2019-04-11 17:41:24 -04:00
Bahram Chechrazy 86483bf961 HBASE-22194 Snapshot unittests fail on Windows due to invalid file path uri 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2019-04-11 11:34:27 -07:00
Jan Hentschel 45ea084e8b
HBASE-22189 Removed remaining usage of StoreFile.getModificationTimeStamp 2019-04-11 12:13:28 +02:00
Duo Zhang cbea2879c4 HBASE-22198 Fix flakey TestAsyncTableGetMultiThreaded 2019-04-11 16:33:39 +08:00
Duo Zhang ae79db959e HBASE-22196 Split TestRestartCluster 2019-04-11 10:58:09 +08:00
huzheng 2e798f87b7 HBASE-22185 RAMQueueEntry#writeToCache should freeBlock if any exception encountered instead of the IOException catch block 2019-04-10 14:06:10 +08:00
Jingyun Tian 8a6b8e81aa HBASE-21965 Fix failed split and merge transactions that have failed to roll back 2019-04-09 20:11:56 +08:00
Duo Zhang b0de9d8391 HBASE-22153 Fix the flaky TestRestartCluster 2019-04-09 18:34:20 +08:00
Duo Zhang 9a1a7eb8c4 HBASE-22188 Make TestSplitMerge more stable 2019-04-09 18:30:25 +08:00
meiyi 0e85571b39 HBASE-22117 Move hasPermission/checkPermissions from region server to master 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-09 17:06:09 +08:00
Guanghao 66fffaa6d0 HBASE-22163 Should not archive the compacted store files when region warmup 2019-04-08 22:20:05 +08:00
Duo Zhang a10ac6f4a1 HBASE-22178 Addendum remove createTableAsync references where we pass null as splitKeys 2019-04-08 09:41:29 +08:00
zhangduo b04b1ecc74 HBASE-22108 Avoid passing null in Admin methods 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-07 21:08:55 +08:00
zhangduo 507af5fe6c HBASE-22180 Make TestBlockEvictionFromClient.testBlockRefCountAfterSplits more stable 2019-04-07 20:57:18 +08:00
chaiwentao 5e03efc558 HBASE-22070 Checking restoreDir in RestoreSnapshotHelper 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-04-06 16:11:04 +02:00
Duo Zhang e35ff3acb7 HBASE-22158 RawAsyncHBaseAdmin.getTableSplits should filter out none default replicas 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-04-05 20:33:54 +08:00
zhangduo 6837a76ba9 HBASE-22133 Forward port HBASE-22073 "/rits.jsp throws an exception if no procedure" to branch-2.2+ 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-04 10:33:24 +08:00
Vladimir Rodionov ae4bfabeaa HBASE-21688 Address WAL filesystem issues 
Amending-Author: Josh Elser <elserj@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-04-03 13:55:48 -04:00
Duo Zhang 1a12b244ec HBASE-22141 Fix TestAsyncDecommissionAdminApi 2019-04-02 18:54:11 +08:00
zhangduo 6fad60232b HBASE-22135 AsyncAdmin will not refresh master address 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-01 09:24:00 +08:00
pingsutw b78c7a4f00 HBASE-22101 AsyncAdmin.isTableAvailable should not throw TableNotFoundException 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-03-30 20:43:12 +08:00
zhangduo 4cf8e3b04c HBASE-22121 AsyncAdmin can not deal with non default meta replica 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-30 20:19:02 +08:00
zhangduo 359cd73e25 HBASE-22099 Backport HBASE-21895 "Error prone upgrade" to branch-2 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-03-30 14:28:15 +08:00
zhangduo b1edddcf89 Revert "HBASE-22073 /rits.jsp throws an exception if no procedure" 
This reverts commit 85c17e711c.
 2019-03-30 09:34:47 +08:00
Sergey Shelukhin 8bceaf7f0b HBASE-22115 HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe 
Signed-off-by: Andrew Purtell <apurtell@apache.org>, stack <stack@apache.org>
 2019-03-29 13:42:20 -07:00
Balazs Meszaros 85c17e711c HBASE-22073 /rits.jsp throws an exception if no procedure 2019-03-29 07:43:40 -07:00
Sakthi 37b2cab98a HBASE-22094: Throw TableNotFoundException if table not exists in AsyncAdmin.compact 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-03-27 20:14:29 +08:00
meiyi 89dbab7c9f HBASE-21911 Move getUserPermissions from regionserver to master 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-27 11:22:57 +08:00
xcang f5b20451db
HBASE-22067 Fix log in StochasticLoadBalancer when balancer is ill-fit for cluster size 2019-03-26 11:42:58 -07:00
zhangduo bb52a6766a HBASE-22074 Should use procedure store to persist the state in reportRegionStateTransition 2019-03-26 18:14:17 +08:00
yaojingyi 1926b86a8c HBASE-21964 unset Quota by Throttle Type 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-26 09:46:01 +08:00
Wei-Chiu Chuang d8d8dcf5fe HBASE-21619. Fix warning message caused by incorrect ternary operator evaluation 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-03-25 20:03:29 +01:00
stack 939a29b41e HBASE-22052 pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications 
This is a reapply of a reverted commit. This commit includes
HBASE-22059 amendment and subsequent ammendments to HBASE-22052.
See HBASE-22052 for full story.

jersey-core is problematic. It was transitively included from hadoop
and polluting our CLASSPATH with an implementation of a 1.x version
of the javax.ws.rs.core.Response Interface from jsr311-api when we
want the javax.ws.rs-api 2.x version.

    M hbase-endpoint/pom.xml
    M hbase-http/pom.xml
    M hbase-mapreduce/pom.xml
    M hbase-rest/pom.xml
    M hbase-server/pom.xml
    M hbase-zookeeper/pom.xml
     Remove redundant version specification (and the odd property define
     done already up in parent pom).
    M hbase-it/pom.xml
    M hbase-rest/pom.xml
     Exclude jersey-core explicitly.

    M hbase-procedure/pom.xml
     Remove redundant version and classifier.

    M pom.xml
     Add jersey-core exclusions to all dependencies that pull it in
     except hadoop-minicluster. mr tests fail w/o the jersey-core
     so let it in for minicluster and then in modules, exclude it
     where it causes damage as in hbase-it.
 2019-03-25 09:28:39 -04:00
Toshihiro Suzuki b11ab1c4cf HBASE-22093 Combine TestRestoreSnapshotFromClientWithRegionReplicas to CloneSnapshotFromClientAfterSplittingRegionTestBase#testCloneSnapshotAfterSplittingRegion 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-03-24 15:08:40 +09:00
zhangduo 8fb8886e5c Revert "HBASE-21965 Fix failed split and merge transactions that have failed to roll back" 
This reverts commit b2a4265992.
 2019-03-24 13:50:39 +08:00
Toshihiro Suzuki 71ea921d85 HBASE-22095 Taking a snapshot fails in local mode 
Signed-off-by: Zach York <zyork@apache.org>
 2019-03-24 13:50:09 +09:00
Jingyun Tian b2a4265992 HBASE-21965 Fix failed split and merge transactions that have failed to roll back 2019-03-22 17:12:34 +08:00
Sakthi d6f23388f5
HBASE-21991: Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-03-21 13:55:33 -07:00
niuyulin ffcc003f85 HBASE-22065 Add listTableDescriptors(List<TableName>) method in AsyncAdmin 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-03-21 22:36:10 +08:00
Jingyun Tian b5fdce7c6b HBASE-22061 SplitTableRegionProcedure should hold the lock of its daughter regions 2019-03-20 14:13:15 +08:00
meiyi 3317f03d41 HBASE-22015 UserPermission should be annotated as InterfaceAudience.Public 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-19 14:55:15 +08:00
zhangduo 2277ca422e HBASE-22040 Add mergeRegionsAsync with a List of region names method in AsyncAdmin 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-03-18 21:05:27 +08:00
Andrew Purtell ea653777e7
HBASE-21926 Profiler servlet 2019-03-17 18:48:16 -07:00
Nihal Jain 04b78bb416 HBASE-20662 Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement 
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-03-15 14:00:52 -04:00
zhangduo 839dad2c9e HBASE-21736 Remove the server from online servers before scheduling SCP for it in hbck 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-03-13 11:07:22 +08:00
meiyi 8fd1b0afe9 HBASE-21977 Skip replay WAL and update seqid when open regions restored from snapshot 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-03-12 21:25:06 +08:00
zhangduo 31b3716e84 HBASE-22001 Polish the Admin interface 
Signed-off-by: stack <stack@apache.org>
 2019-03-11 14:43:12 +08:00
Jack Bearden cce40e14bc HBASE-20918 Re-enable TestRpcHandlerException 
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 65149bd8fc)
 2019-03-08 11:25:32 -06:00
zhangduo d084423b52 HBASE-22007 Add restoreSnapshot and cloneSnapshot with acl methods in AsyncAdmin 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-03-08 18:19:11 +08:00
ramkrishna 82618c439d HBASE-21874 Bucket cache on Persistent memory 
Signed-off-by: Anoop John <anoop.hbase@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit f67f207965)
 2019-03-07 08:53:31 -06:00
stack 89c8056993 HBASE-22006 Fix branch-2.1 findbugs warning; causes nightly show as failed. 
(cherry picked from commit 764adaad34)
 2019-03-07 08:52:55 -06:00
Sean Busbey ee30872dcf Revert "SE-22006 Fix branch-2.1 findbugs warning; causes nightly show as failed." 
This reverts commit 764adaad34.

bad commit message
 2019-03-07 08:52:29 -06:00
Sean Busbey a5c26653f9 Revert "HBASE-21874 Bucket cache on Persistent memory" 
This reverts commit f67f207965.

bad signed off by line
 2019-03-07 08:52:13 -06:00
ramkrishna f67f207965 HBASE-21874 Bucket cache on Persistent memory 
Signed-off-by: Anoop John <anoop.hbase@gmail.com>, Sean Busbey
<busbey@apache.org>
 2019-03-07 10:52:18 +05:30
stack 764adaad34 SE-22006 Fix branch-2.1 findbugs warning; causes nightly show as failed. 2019-03-06 16:01:18 -08:00
subrat.mishra 2f9ab7d95a HBASE-21871 Added support to specify a peer table name in VerifyReplication tool 
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
 2019-03-06 23:29:08 +09:00
maoling 80ae8c9f97 HBASE-21949 Fix flaky test TestHBaseTestingUtility.testMiniZooKeeperWithMultipleClientPorts 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-06 18:05:33 +08:00
Duo Zhang 506e01a064 HBASE-21094 Remove the explicit timeout config for TestTruncateTableProcedure 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-04 15:34:57 +08:00
Guanghao Zhang f7aca3fa06 Revert "HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure" 
This reverts commit cc6fc1f04f.
 2019-03-04 15:32:13 +08:00
zhangduo ab8d9e736f HBASE-21093 Move TestCreateTableProcedure.testMRegions to a separated file 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-04 15:20:06 +08:00
Guanghao Zhang 6f8f0fa533 Revert "HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file" 
This reverts commit b318311dfd.
 2019-03-04 15:19:22 +08:00
zhangduo cc924513ac HBASE-21980 Fix typo in AbstractTestAsyncTableRegionReplicasRead 2019-03-03 08:29:47 +08:00
meiyi 0d882bbc2b HBASE-21974 Change Admin#grant/revoke parameter from UserPermission to user and Permission 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-02 23:47:56 +08:00
Syeda 5c03df5e50 HBASE-21487 Concurrent modify table ops can lead to unexpected results 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-02 13:07:35 +08:00
Duo Zhang 8677031707 HBASE-21976 Deal with RetryImmediatelyException for batching request 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-02 12:14:32 +08:00
Reid Chan 6de61d6bee HBASE-21481 [acl] Superuser's permissions should not be granted or revoked by any non-su global admin 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-01 23:42:11 +08:00
Duo Zhang 1138f72c18 HBASE-21082 Reimplement assign/unassign related procedure metrics 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-03-01 21:08:44 +08:00
Jingyun Tian ebab2fc913 HBASE-21934 RemoteProcedureDispatcher should track the ongoing dispatched calls 2019-03-01 17:55:29 +08:00
Xu cang 3c6ac6d3f8
HBASE-21952 Fix unit test: TestClientOperationInterrupt#testInterrupt50Percent 2019-02-28 22:19:44 -08:00
Guanghao Zhang c2e5f7b505 HBASE-21967 Split TestServerCrashProcedure and TestServerCrashProcedureWithReplicas 2019-02-28 17:46:47 +08:00
Guanghao Zhang e412e036e1 HBASE-20724 Sometimes some compacted storefiles are still opened after region failover 2019-02-28 09:51:29 +08:00
meiyi ea0675eb96 HBASE-21820 Implement CLUSTER quota scope 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-27 18:24:03 +08:00
Duo Zhang 11624c63e3 HBASE-21961 Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we tried to locate before a non empty row 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-27 17:55:09 +08:00
Duo Zhang 80830ee555 HBASE-21943 The usage of RegionLocations.mergeRegionLocations is wrong for async client 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-26 11:22:11 +08:00
xuqinya 3062618ac9 HBASE-21942 [UI] requests per second is incorrect in rsgroup page(rsgroup.jsp) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-02-25 16:24:49 +01:00
Wellington Chevreuil a8af0e5894 HBASE-21505 - proposal for a more consistent report on status 
Signed-off-by: Jingyun Tian <tianjy@apache.org>
 2019-02-25 10:07:51 +08:00
zhangduo d7c5d73de0 HBASE-21944 Validate put for batch operation 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-24 20:20:16 +08:00
zhangduo 1568aad302 HBASE-21945 Maintain the original order when sending batch request 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-24 17:56:16 +08:00
Guanghao Zhang 607ac735c4 HBASE-21922 BloomContext#sanityCheck may failed when use ROWPREFIX_DELIMITED bloom filter 2019-02-23 23:29:53 +08:00
Duo Zhang 714e5225e8 HBASE-21929 The checks at the end of TestRpcClientLeaks are not executed 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-02-23 20:57:10 +08:00
meiyi 9e45752d6e HBASE-21783 Support exceed user/table/ns throttle quota if region server has available quota 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-22 17:43:40 +08:00
Duo Zhang 761aef6d9d HBASE-20587 Replace Jackson with shaded thirdparty gson 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-02-22 16:40:45 +08:00
meiyi 61621a33ce HBASE-21938 Add a new ClusterMetrics.Option SERVERS_NAME to only return the live region servers's name without metrics 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-22 10:45:35 +08:00
Toshihiro Suzuki 04caf89e80 HBASE-21867 Support multi-threads in HFileArchiver 2019-02-21 19:05:45 +09:00
zhangduo 164c250691 HBASE-21927 Always fail the locate request when error occur 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-02-20 19:48:32 +08:00
Guanghao Zhang bcbbfaebc4 HBASE-21932 Use Runtime.getRuntime().halt to terminate regionserver when abort timeout 2019-02-20 15:55:45 +08:00
zhangduo f198311556 HBASE-21928 Deprecated HConstants.META_QOS 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-02-20 14:16:54 +08:00
zhangduo 4b36de3161 HBASE-21930 Deal with ScannerResetException when opening region scanner 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-02-20 11:04:15 +08:00
Josh Elser c3312b9c53 HBASE-21915 Make FileLinkInputStream implement CanUnbuffer 
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
 2019-02-19 16:17:28 -05:00
Xiang Li 17736e0a87
HBASE-21866 Do not move the table to null rsgroup when creating an existing table 2019-02-19 11:04:42 -08:00
Nihal Jain f671703c7f HBASE-17942 Disable region splits and merges per table 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-19 07:20:34 +08:00
lujie f5ef0442b0
HBASE-21740 Fix NPE while shutting down RS 2019-02-18 02:55:17 -08:00
Daisuke Kobayashi 98c6ab68c1 HBASE-21899 Fix missing variables for slf4j Logger 
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
 2019-02-17 20:15:44 +09:00
zhangduo d88a87fd04 HBASE-21909 Validate the put instance before executing in AsyncTable.put method 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-02-16 18:06:39 +08:00
Duo Zhang 48f2ef432b HBASE-21910 The nonce implementation is wrong for AsyncTable 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-15 20:24:48 +08:00
Guanghao Zhang 1650381e96 HBASE-21814 Remove the TODO in AccessControlLists#addUserPermission 2019-02-15 19:36:30 +08:00
Duo Zhang a24ef97073 HBASE-21875 Change the retry logic in RSProcedureDispatcher to 'retry by default, only if xxx' 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-15 11:05:35 +08:00
Duo Zhang ecbcf6ca2f HBASE-21890 Use execute instead of submit to submit a task in RemoteProcedureDispatcher 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-02-14 14:19:11 +08:00
Duo Zhang 33bdf09f77 HBASE-21888 Add a isClosed method to AsyncConnection 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-14 11:41:30 +08:00
Sean Busbey cefaecdc83 HBASE-21884 avoid autoboxing in ugi ref counting for secure bulk load 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 4273e42ce5)
 2019-02-13 13:53:44 -06:00
zhangduo beb7d01092 HBASE-21859 Add clearRegionLocationCache method for AsyncConnection 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-02-13 10:07:43 +08:00
Sakthi 9853f788e8 HBASE-21780: Avoid a wide line on the RegionServer webUI for many ZooKeeper servers 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-02-12 11:00:09 +01:00
huzheng df8fa5ef23 HBASE-18484 VerifyRep by snapshot does not work when Yarn/SourceHBase/PeerHBase located in three different HDFS clusters 2019-02-12 11:06:11 +08:00
zhangduo c3f0466336 HBASE-21857 Addendum fix broken UTs 2019-02-11 10:26:34 +08:00
Sakthi 92eb920eca
HBASE-21800: RegionServer aborted due to NPE from MetaTableMetrics coprocessor 
Have included code refactoring in MetaTableMetrics & LossyCounting
 2019-02-08 10:44:29 -08:00
zhangduo 94c9aecea2 HBASE-21857 Do not need to check clusterKey if replicationEndpoint is provided when adding a peer 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-02-08 19:48:07 +08:00
kpalanisamy fceafde0fc HBASE-21816 : Print source cluster replication config directory 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
 2019-02-08 14:10:08 +09:00
zhangduo c5b80ea2d4 HBASE-21843 RegionGroupingProvider breaks the meta wal file name pattern which may cause data loss for meta region 2019-02-07 08:44:21 +08:00
zhangduo d66e683216 HBASE-21838 Create a special ReplicationEndpoint just for verifying the WAL entries are fine 2019-02-06 18:30:10 +08:00
Beluga Behr 1492bde036 HBASE-19616 Review of LogCleaner Class 2019-02-05 10:30:15 -08:00
Nihal Jain 4b2c5ae632 HBASE-21795 Client application may get stuck (time bound) if a table modify op is called immediately after split op 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-02-04 20:50:22 +08:00
zhangduo 2c6ab0c047 HBASE-21840 TestHRegionWithInMemoryFlush fails with NPE 2019-02-04 19:26:04 +08:00
zhangduo 54a8cc87c6 HBASE-21829 Use FutureUtils.addListener instead of calling whenComplete directly 2019-02-03 15:29:20 +08:00
zhangduo b40aa4b099 HBASE-21833 Use NettyAsyncFSWALConfigHelper.setEventLoopConfig to prevent creating too many netty event loop when executing TestHRegion 2019-02-03 15:00:14 +08:00
zhangduo ed60532543 HBASE-21828 Make sure we do not return CompletionException when locating region 2019-02-02 21:14:09 +08:00
Sergey Shelukhin 9c8aca73ed HBASE-21811 region can be opened on two servers due to race condition with procedures and server reports 
The original fix is provided by Sergey Shelukhin, the UT is added by Duo Zhang

Amending-Author: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-02-02 17:39:55 +08:00
huzheng 2b5dd388cd HBASE-21764 Size of in-memory compaction thread pool shoud be configurable 2019-02-01 10:34:07 +08:00
Nihal Jain 70fb6951d8 HBASE-21644 Modify table procedure runs infinitely for a table having region replication > 1 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-01-31 19:34:36 +08:00
Nihal Jain 3ef6366c4f HBASE-21404 Master/RS navbar active state does not work 
Signed-off-by: Jingyun Tian <tianjy@apache.org>
 2019-01-31 16:34:33 +08:00
meiyi ec9029f77e HBASE-21739 Move grant/revoke from regionserver to master 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-30 19:03:43 +08:00
meiyi 32f3745e63 HBASE-21733 SnapshotQuotaObserverChore should only fetch space quotas 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-30 17:57:59 +08:00
wanghuan 69c331efed HBASE-21699 Fixed create table failed when using SPLITS_FILE => 'splits.txt' 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-30 15:57:21 +08:00
Pankaj 529eb47281 HBASE-21535 Zombie Master detector is not working 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-30 15:01:46 +08:00
Guanghao Zhang 16665b6e93 HBASE-21799 Update branch-2 version to 2.3.0-SNAPSHOT 2019-01-29 21:53:21 +08:00
zhangduo f359d260d3 HBASE-21789 Rewrite MetaTableAccessor.multiMutate with Table.coprocessorService 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-27 11:37:11 +08:00
zhangduo 2d4819dbed HBASE-21782 LoadIncrementalHFiles should not be IA.Public 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-27 11:26:00 +08:00
Duo Zhang 1ac090b3f0 HBASE-21762 Move some methods in ClusterConnection to Connection 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-01-25 21:14:07 +08:00
meiyi 1e7f306e17 HBASE-21713 Support set region server throttle quota 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-25 15:47:32 +08:00
Jingyun Tian 3e72cb73ae HBASE-21729 Extract ProcedureCoordinatorRpcs and ProcedureMemberRpcs from CoordinatedStateManager 2019-01-25 14:28:33 +08:00
Wellington Chevreuil aae52f787c HBASE-20215 - Rename CollectionUtils to ConcurrentMapUtils 2019-01-24 20:55:55 -08:00
zhangduo de31d56c2d HBASE-21770 Should deal with meta table in HRegionLocator.getAllRegionLocations 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-25 11:05:29 +08:00
zhangduo 67ba050fce HBASE-21761 Align the methods in RegionLocator and AsyncTableRegionLocator 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-24 15:21:26 +08:00
Tommy Li 8a7dd2de24 HBASE-21720 : metric to measure how actions are distributed to servers within a MultiAction 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2019-01-23 17:53:59 -08:00
Nihal Jain 41e0362fd5
HBASE-21475 Put mutation (having TTL set) added via co-processor is retrieved even after TTL expires 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-01-23 16:57:35 -08:00
Allan Yang d80556ec85 HBASE-21754 ReportRegionStateTransitionRequest should be executed in priority executor 2019-01-23 21:08:13 +08:00
Duo Zhang 6c57050af6 HBASE-21753 Support getting the locations for all the replicas of a region 2019-01-23 18:10:01 +08:00
Sergey Shelukhin 96b98da19d HBASE-21575 : memstore above high watermark message is logged too much 2019-01-22 15:28:29 -08:00
huzheng 4eba6b3656 HBASE-21750 Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized 2019-01-22 20:21:49 +08:00
Nihal Jain 5b99a34a31
HBASE-21749 RS UI may throw NPE and make rs-status page inaccessible with multiwal and replication 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-01-21 17:26:17 -08:00
huzheng c679a76845 HBASE-21738 Remove all the CSLM#size operation in our memstore because it's an quite time consuming. 2019-01-21 18:01:16 +08:00
zhangduo e0d4915f75 HBASE-21746 Fix two concern cases in RegionMover 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-20 08:57:16 +08:00
Duo Zhang ff272e8683 HBASE-21726 Add getAllRegionLocations method to AsyncTableRegionLocator 2019-01-19 11:14:40 +08:00
zhangduo 07484db1ff HBASE-19695 Handle disabled table for async client 
Signed-off-by: tianjingyun <tianjy1990@gmail.com>
 2019-01-17 18:34:30 +08:00
Beluga Behr fffad56dd6 HBASE-20209 Do Not Use Both Map containsKey and get Methods in Replication Sink 
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 3e92cd5771)
 2019-01-16 15:16:30 -07:00
Duo Zhang ebf4fe3bb9 HBASE-21710 Add quota related methods to the Admin interface 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-16 21:10:27 +08:00
huzheng fdfbd9a21c HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case 2019-01-16 09:47:27 +08:00
tianjingyun 87d4ab8669 HBASE-21647 Add status track for splitting WAL tasks 2019-01-15 16:39:02 -08:00
Pankaj 11a0793d72 HBASE-21639 maxHeapUsage value not read properly from config during EntryBuffers initialization 2019-01-15 12:18:53 -08:00
Pankaj a9f40d3bf7 HBASE-21595, Print thread's information and stack traces when RS is aborting forcibly 2019-01-15 12:14:40 -08:00
Sakthi dc50b570c6 HBASE-21225: Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE' 
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-01-14 18:01:33 -07:00
zhangduo e7113edddf HBASE-21705 Should treat meta table specially for some methods in AsyncAdmin 2019-01-13 13:41:01 +08:00
Duo Zhang 9ec84c235f HBASE-21704 The implementation of DistributedHBaseCluster.getServerHoldingRegion is incorrect 2019-01-11 21:20:50 +08:00
Duo Zhang a09dffd106 HBASE-21663 Add replica scan support 2019-01-11 14:19:17 +08:00
Duo Zhang 8b9bfa0df5 HBASE-21580 Support getting Hbck instance from AsyncConnection 2019-01-11 11:26:48 +08:00
Nihal Jain d394d84420 HBASE-21297 ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-10 22:15:49 +08:00
Guanghao Zhang 2cf13d04a1 HBASE-21695 Fix flaky test TestRegionServerAbortTimeout 2019-01-10 10:23:47 +08:00
Guanghao Zhang f0c0f3e6ec HBASE-21691 Fix flaky test TestRecoveredEdits 2019-01-10 10:11:31 +08:00
meiyi 14fb3af5fb HBASE-21694 Add append_peer_exclude_tableCFs and remove_peer_exclude_tableCFs shell commands 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-09 22:22:47 +08:00
Sergey Shelukhin dfa86404dd HBASE-21614 RIT recovery with ServerCrashProcedure doesn't account for all regions 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-01-09 20:03:30 +08:00
Nihal Jain 26e02e1c0b HBASE-21645 Perform sanity check and disallow table creation/modification with region replication < 1 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-09 11:05:16 +08:00
Jingyun Tian f02ac310d2 HBASE-21588 Procedure v2 wal splitting implementation 2019-01-08 17:26:58 +08:00
zhangduo 348c2dfe9b HBASE-21683 Reset readsEnabled flag after successfully flushing the primary region 2019-01-08 15:51:48 +08:00
zhangduo d5fff9c256 HBASE-21682 Support getting from specific replica 2019-01-08 10:04:08 +08:00
meiyi c28e03e5df HBASE-21159 Add shell command to switch throttle on or off 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-08 09:39:22 +08:00
zhangduo 77ca660389 HBASE-17356 Add replica get support 2019-01-03 08:38:53 +08:00
Guanghao Zhang ba9a49b8a5 HBASE-21659 Avoid to load duplicate coprocessors in system config and table descriptor 2019-01-02 11:45:21 +08:00
Guanghao Zhang a237d97f72 HBASE-21660 Apply the cell to right memstore for increment/append operation 2019-01-01 17:38:37 +08:00
huzheng f60e8b0b95 HBASE-21642 CopyTable by reading snapshot and bulkloading will save a lot of time 2018-12-27 18:50:10 +08:00
Guanghao Zhang cbc815cfb7 HBASE-21643 Introduce two new region coprocessor method and deprecated postMutationBeforeWAL 2018-12-27 18:29:38 +08:00
Guanghao Zhang e24d0e0ae0 HBASE-21640 Remove the TODO when increment zero 2018-12-26 21:52:26 +08:00
Andrey Elenskiy e7bd69f9e9 HBASE-21545 NEW_VERSION_BEHAVIOR breaks Get/Scan with specified columns 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sakthi
 2018-12-23 22:02:16 -08:00
Duo Zhang ad285eae4f HBASE-21635 Use maven enforcer to ban imports from illegal packages 2018-12-24 11:30:13 +08:00
Guanghao Zhang 24f740c3d9 HBASE-21621 Reversed scan does not return expected number of rows 
The unit test is contributed by Nihal Jain
 2018-12-23 16:26:39 +08:00
Sakthi b396051153 HBASE-21631: list_quotas should print human readable values for LIMIT 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-12-22 22:05:32 +08:00
openinx 406f44219f HBASE-21620 Problem in scan query when using more than one column prefix filter in some cases 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-12-21 15:21:34 -08:00
huzheng 46ba4591a4 HBASE-21401 Sanity check when constructing the KeyValue 2018-12-21 18:50:27 +08:00
Guanghao Zhang 1dd137b148 HBASE-21618 Scan with the same startRow(inclusive=true) and stopRow(inclusive=false) returns one result 2018-12-21 09:50:23 +08:00
Pankaj f47761f91c HBASE-21610, numOpenConnections metric is set to -1 when zero server channel exist 2018-12-20 16:36:23 -08:00
Jingyun Tian 83e12153d1 HBASE-21565 Delete dead server from dead server list too early leads to concurrent Server Crash Procedures(SCP) for a same server 2018-12-19 18:42:23 +08:00
Guanghao Zhang d2832c1708 HBASE-21514 Refactor CacheConfig 2018-12-19 13:49:58 +08:00
BELUGA BEHR fc7ca8a2ef
HBASE-21492 CellCodec Written To WAL Before It's Verified 2018-12-18 17:45:37 -08:00
xuqinya 99de534cc4 HBASE-21592 quota.addGetResult(r) throw NPE 
Signed-off-by: huzheng <openinx@gmail.com>
 2018-12-18 16:51:46 +08:00
stack ee214cbe40 HBASE-21589 TestCleanupMetaWAL fails 2018-12-17 09:31:35 -08:00
huzheng 56e7489625 HBASE-21520 TestMultiColumnScanner cost long time when using ROWCOL bloom type 2018-12-15 21:10:48 +08:00
Sean Busbey aa36c3f6b6 HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. (addendum) 
(cherry picked from commit a284c493b127fbf785fbee401f0854ad1979eb26)
 2018-12-14 15:30:31 -06:00
meiyi c494e9ff20 HBASE-21578 Fix wrong throttling exception for capacity unit 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-12-14 18:22:34 +08:00
Lars Hofhansl 11193d7cc1 HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. 2018-12-13 11:56:39 -08:00
huzheng 9b139ca011 HBASE-21582 If call HBaseAdmin#snapshotAsync but forget call isSnapshotFinished, then SnapshotHFileCleaner will skip to run every time 2018-12-12 21:53:37 +08:00
Josh Elser b8405bfea4 HBASE-21568 Use CacheConfig.DISABLED where we don't expect to have blockcache running 
This includes removing the "old way" of disabling blockcache in favor of the
new API.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-12-11 11:37:35 -05:00
zhangduo 9981e3f843 HBASE-21570 Add write buffer periodic flush support for AsyncBufferedMutator 2018-12-11 15:29:39 +08:00
Guanghao Zhang 0094335c52 HBASE-21560 Return a new TableDescriptor for MasterObserver#preModifyTable to allow coprocessor modify the TableDescriptor 2018-12-08 16:03:30 +08:00
Guanghao Zhang 7b5b99e243 HBASE-21554 Show replication endpoint classname for replication peer on master web UI 2018-12-07 16:57:52 +08:00
Allan Yang f86d311f76 HBASE-21413 Empty meta log doesn't get split when restart whole cluster 
Signed-off-by: stack <stack@apache.org>
 2018-12-06 21:20:55 -08:00
David Manning 2ebf5ab02f Backport "HBASE-21126 Add ability for HBase Canary to ignore a configurable number of ZooKeeper down nodes" to branch-2.0 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-12-06 21:20:51 -08:00
huzheng 5cb8c3e9c7 HBASE-21559 The RestoreSnapshotFromClientTestBase related UT are flaky 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-12-07 08:25:49 +08:00
huzheng 1a1a65b565 HBASE-21551 Memory leak when use scan with STREAM at server side 2018-12-06 11:20:38 +08:00
Duo Zhang 96f8e0cbe9 HBASE-21534 TestAssignmentManager is flakey 2018-12-06 09:54:18 +08:00
Duo Zhang 7e8ce1c5cc HBASE-21550 Add a new method preCreateTableRegionInfos for MasterObserver which allows CPs to modify the TableDescriptor 2018-12-06 08:30:37 +08:00
Duo Zhang 5648337545 HBASE-21541 Move MetaTableLocator.verifyRegionLocation to hbase-rsgroup module 2018-12-04 22:27:49 +08:00
Peter Somogyi 4118bc688e HBASE-21479 Individual tests in TestHRegionReplayEvents class are failing 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-12-01 09:48:19 +01:00
Peter Somogyi 9d5df33be5 HBASE-21518 TestMasterFailoverWithProcedures is flaky 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-11-30 09:13:38 +01:00
huzheng 1be42994a5 HBASE-21504 If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived. 2018-11-30 10:03:46 +08:00
Artem Ervits ad8fbf3fd3 HBASE-18735 Provide an option to kill a MiniHBaseCluster without waiting on shutdown 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-11-29 17:05:04 -05:00
meiyi aaafafe63b HBASE-21300 Fix the wrong reference file path when restoring snapshots for tables with MOB columns 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-29 13:57:33 +08:00
Duo Zhang 5688944721 HBASE-21517 Move the getTableRegionForRow method from HMaster to TestMaster 2018-11-28 09:41:45 +08:00
lixiaobao 3c043440a8 HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
 2018-11-27 20:58:24 +08:00
zhangduo beeb38d4dc Revert "HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method" 
This reverts commit 6aa57f71f0.
 2018-11-27 20:58:14 +08:00
lixiaobao 6aa57f71f0 HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
 2018-11-27 20:31:42 +08:00
Guanghao Zhang 958e84d023 HBASE-21498 Master OOM when SplitTableRegionProcedure new CacheConfig and instantiate a new BlockCache 2018-11-27 14:23:27 +08:00
Duo Zhang 33c4da5558 HBASE-21508 Ignore the reportRegionStateTransition call from a dead server 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-26 14:32:33 +08:00
Ted Yu 7f02645741 HBASE-21511 Remove in progress snapshot check in SnapshotFileCache#getUnreferencedFiles 2018-11-25 18:22:53 -08:00
Ted Yu 1731666b43 HBASE-21387 Addendum fix TestSnapshotFileCache 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-11-24 17:00:31 +08:00
huzheng cc26c8716f HBASE-21387 Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files 2018-11-23 11:51:58 +08:00
Duo Zhang 248b8a6f56 HBASE-21503 Replication normal source can get stuck due potential race conditions between source wal reader and wal provider initialization threads. 
Found and analysed by Wellington Chevreuil
 2018-11-21 17:59:06 +08:00
meiyi d590d6e472 HBASE-21034 Add new throttle type: read/write capacity unit 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-21 09:50:07 +08:00
Duo Zhang 6a64811f44 HBASE-21490 WALProcedure may remove proc wal files still with active procedures 
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-11-19 08:21:09 -08:00
zhangduo 7d16cf0977 HBASE-21480 Taking snapshot when RS crashes prevent we bring the regions online 2018-11-18 21:07:27 +08:00
zhangduo 37540b541d HBASE-21485 Add more debug logs for remote procedure execution 2018-11-16 14:47:24 +08:00
Reid Chan 6ab0fbb1f1 HBASE-21255 [acl] Refactor TablePermission into three classes (Global, Namespace, Table) 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-11-15 11:45:24 +08:00
zhangduo 17055cdc70 HBASE-21472 Should not persist the dispatched field for RegionRemoteProcedureBase 2018-11-15 09:41:43 +08:00
huzheng a765859131 HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags 2018-11-14 23:13:04 +08:00
zhangduo fbf5e9e0c4 HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors 2018-11-14 18:55:27 +08:00
zhangduo a44515aea7 Revert "HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors" 
This reverts commit a27eea0807.
 2018-11-13 22:24:02 +08:00
zhangduo f8970bdf04 Revert "HBASE-21465 Addendum change TestReportRegionStateTransitionRetry to MediumTests" 
This reverts commit 3153aae128.
 2018-11-13 22:23:25 +08:00
Duo Zhang 3153aae128 HBASE-21465 Addendum change TestReportRegionStateTransitionRetry to MediumTests 2018-11-13 14:59:48 +08:00
Duo Zhang a27eea0807 HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors 2018-11-13 11:52:12 +08:00
Duo Zhang ea3b2dfaeb HBASE-21463 The checkOnlineRegionsReport can accidentally complete a TRSP 2018-11-13 11:17:52 +08:00
tedyu 61f1d9735b HBASE-21466 WALProcedureStore uses wrong FileSystem if wal.dir is not under rootdir 2018-11-12 17:02:45 -08:00
huzheng 98ad5ca6e5 Revert "HBASE-21401 Sanity check in BaseDecoder#parseCell" 
This reverts commit b6d32e8a10.
 2018-11-12 11:42:04 +08:00
Duo Zhang f1d9f59bfe HBASE-21376 Add some verbose log to MasterProcedureScheduler 2018-11-10 17:17:24 +08:00
jingyuntian 6f602db5f2 HBASE-21410 A helper page that help find all problematic regions and procedures 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-09 14:02:57 +08:00
Guanghao Zhang f936a1045d Revert "HBASE-21410 A helper page that help find all problematic regions and procedures" 
This reverts commit a2f650a0ec.

missing sign-off.
 2018-11-09 14:02:38 +08:00
jingyuntian a2f650a0ec HBASE-21410 A helper page that help find all problematic regions and procedures 2018-11-09 13:56:43 +08:00
Ben Lau 13b68abb58
HBASE-21439 RegionLoads aren't being used in RegionLoad cost functions 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
 2018-11-08 14:08:52 -08:00
Esteban Gutierrez dcdebbffdc
HBASE-20604 ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-11-08 10:22:11 -08:00
huzheng b6d32e8a10 HBASE-21401 Sanity check in BaseDecoder#parseCell 2018-11-08 20:14:03 +08:00
tedyu 399fbb2aa6 HBASE-21247 Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers 2018-11-06 12:35:31 -08:00
Allan Yang d544b7adea HBASE-21421 Do not kill RS if reportOnlineRegions fails 2018-11-06 14:56:33 +08:00
Michael Stack a0bba8db46 HBASE-21425 2.1.1 fails to start over 1.x data; namespace not assigned 2018-11-03 09:45:57 -07:00
zhangduo 4f22397ad4 HBASE-21351 The force update thread may have race with PE worker when the procedure is rolling back 2018-11-03 08:24:25 +08:00
jingyuntian 1bd98bb608
HBASE-21407 Resolve NPE in backup Master UI 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-11-02 11:46:47 -07:00
zhangduo 943f65f3b5 HBASE-21422 NPE in TestMergeTableRegionsProcedure.testMergeWithoutPONR 2018-11-02 20:55:36 +08:00
Beluga Behr 3d3dd04651 HBASE-19682 Use Collections.emptyList() For Empty List Values 2018-11-01 08:59:14 -07:00
jingyuntian b9c9f1d61f HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService (addendum for branch-2) 2018-11-01 18:05:01 +08:00
Guanghao Zhang 2d0810a006 HBASE-21388 No need to instantiate MemStoreLAB for master which not carry table 2018-11-01 16:20:49 +08:00
tianjingyun 857abfb082
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-29 20:56:11 -07:00
zhangduo eb0f9e15d1 HBASE-21375 Revisit the lock and queue implementation in MasterProcedureScheduler 2018-10-29 20:01:53 +08:00
Guanghao Zhang 6fbd70a611 HBASE-21325 Force to terminate regionserver when abort hang in somewhere 2018-10-29 10:12:21 +08:00
Duo Zhang 4ded55702d HBASE-21391 RefreshPeerProcedure should also wait master initialized before executing 2018-10-26 21:45:03 +08:00
Guanghao Zhang f71768308c HBASE-21385 HTable.delete request use rpc call directly instead of AsyncProcess 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-25 16:22:18 -07:00
Michael Stack 8c1b534ee8 HBASE-21372) Set hbase.assignment.maximum.attempts to Long.MAX 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-10-24 09:07:24 -07:00
xcang f3da04a552
HBASE-21349 Do not run CatalogJanitor or Nomalizer when cluster is shutting down 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-23 14:44:45 -07:00
mazhenlin 390c3227e2
HBASE-21342 FileSystem in use may get closed by other bulk load call in secure bulkLoad 
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
 2018-10-23 15:52:05 -05:00
xcang d4ce9e534e
HBASE-21338 Warn if balancer is an ill-fit for cluster size 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-23 13:23:03 -07:00
huzheng bf2ce5488d HBASE-21356 bulkLoadHFile API should ensure that rs has the source hfile's write permissionls 2018-10-23 10:25:06 +08:00
zhangduo 328555a374 HBASE-21336 Addendum remove unused code in HBTU 2018-10-22 20:27:11 +08:00
huzheng ff2b332aef HBASE-21355 (addendum) replace the expensive reload storefiles with reading the merge result of compacted storefiles and current storefiles 2018-10-22 19:33:36 +08:00
zhangduo 0cefe7312c HBASE-21336 Simplify the implementation of WALProcedureMap 2018-10-22 18:36:16 +08:00
zhangduo 69e65f1bd7 HBASE-21334 TestMergeTableRegionsProcedure is flakey 2018-10-22 14:18:59 +08:00
huzheng 6dce9cd218 HBASE-21355 HStore's storeSize is calculated repeatedly which causing the confusing region split 2018-10-22 10:21:57 +08:00
Toshihiro Suzuki 46fe6c0e92 HBASE-21200 Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner. 2018-10-20 08:35:41 +09:00
Mike Drob 444f15785c HBASE-21073 Redo concept of maintenance mode 
Instead of being an ephemeral state set by hbck, maintenance mode is now
an explicit toggle set by either configuration property or environment
variable. In maintenance mode, master will host system tables and not
assign any user-space tables to RSs. This gives operators the ability to
affect repairs to meta table with fewer moving parts.
 2018-10-19 10:04:35 -05:00
jingyuntian 4a609db30c
HBASE-21269　Forward-port HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign 2018-10-18 06:22:22 -07:00
haxiaolin 20a9b8ad00 HBASE-21055 NullPointerException when balanceOverall() but server balance info is null 
Signed-off-by: huzheng <openinx@gmail.com>
 2018-10-18 14:06:08 +08:00
Duo Zhang d7d3beb6bc HBASE-21330 ReopenTableRegionsProcedure will enter an infinite loop if we schedule a TRSP at the same time 2018-10-18 11:34:43 +08:00
tedyu 3563f48900 HBASE-21281 Update bouncycastle dependency - addendum adds dependency for hbase-server module 2018-10-17 16:00:28 -07:00
Michael Stack f32d3e1e2c
HBASE-21327 Fix minor logging issue where we don't report servername if no associated SCP 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-10-17 09:38:26 -07:00
Michael Stack c67f7f14e2
HBASE-21320 [canary] Cleanup of usage and add commentary 
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
 2018-10-16 22:19:34 -07:00
zhangduo 1a0e1039a2 HBASE-21311 Split TestRestoreSnapshotFromClient 2018-10-17 11:18:34 +08:00
subrat.mishra 6511dae07a
HBASE-21263 Mention compression algorithm along with other storefile details 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2018-10-16 12:47:11 -07:00
zhangduo 26292ab3d6 HBASE-21310 Split TestCloneSnapshotFromClient 2018-10-16 15:33:43 +08:00
zhangduo c79927bc22 HBASE-21278 Do not rollback successful sub procedures when rolling back a procedure 2018-10-16 15:12:55 +08:00
Andrew Purtell 3097bbc730
HBASE-21266 Not running balancer because processing dead regionservers, but empty dead rs list 2018-10-15 22:28:16 -07:00
Guanghao Zhang 5a6b68012d HBASE-21290 No need to instantiate BlockCache for master which not carry table 2018-10-15 17:25:07 +08:00
haxiaolin 287697bfdf HBASE-21260 The whole balancer plans might be aborted if there are more than one plans to move a same region 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-10-15 15:54:25 +08:00
zhangduo f01ea9b094 HBASE-21309 Increase the waiting timeout for TestProcedurePriority 2018-10-15 15:27:01 +08:00
Michael Stack 9d6afa9d72 HBASE-21299 List counts of actual region states in master UI tables section 2018-10-12 11:02:16 -07:00
Guanghao Zhang 2ab7e13dfa HBASE-21289 Remove the log "'hbase.regionserver.maxlogs' was deprecated." in AbstractFSWAL 2018-10-12 20:38:19 +08:00
Duo Zhang 9da4c1393d HBASE-21254 Need to find a way to limit the number of proc wal files 2018-10-12 11:05:21 +08:00
Josh Elser 33d72e4db4 HBASE-21281 Upgrade bouncycastle to latest 
BC 1.47 introduced some incompatible API changes which came in via
a new Maven artifact. We don't use any changed API in HBase. This
also removes some unnecessary dependencies on bcprov in other
modules (presumably, they are vestiges)

Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tedyu@apache.org>
 2018-10-11 10:53:02 -04:00
Mike Drob 81adb704c8 HBASE-21287 Allow configuring test master initialization wait time. 2018-10-11 09:50:37 -05:00
Guanghao Zhang 91aceeb28d HBASE-21251 Refactor RegionMover 2018-10-10 15:14:57 +08:00
Michael Stack 701fe7a0c8 HBASE-21280 Add anchors for each heading in UI 
Signed-off-by: Ted Yu <tedyu@apache.org>
 2018-10-09 22:49:54 -07:00
Duo Zhang 17bd935301 HBASE-21250 Addendum remove unused modification in hbase-server module 2018-10-08 14:56:25 +08:00
zhangduo e1c89e5f8e HBASE-21250 Refactor WALProcedureStore and add more comments for better understanding the implementation 2018-10-07 17:12:27 +08:00
Wellington Chevreuil 9e3f3fdc1f HBASE-21185 - WALPrettyPrinter: Additional useful info to be printed by wal printer tool, for debugability purposes 
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-10-04 03:31:15 -07:00
Xu Cang 8a5537b5f5
HBASE-18549 Add metrics for failed replication queue recovery 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-10-01 18:39:03 -07:00
Xu Cang 44b01d5908
HBASE-19275 TestSnapshotFileCache never worked properly 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-10-01 17:12:18 -07:00
zhangduo 1728a20be2 HBASE-21248 Implement exponential backoff when retrying for ModifyPeerProcedure 2018-09-29 13:35:04 +08:00
Nihal Jain 3d1a2dbe68
HBASE-21196 HTableMultiplexer clears the meta cache after every put operation 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 16:35:53 -07:00
Kiran Kumar Maturi f9e18cf31b
HBASE-20857 balancer status tag in jmx metrics 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 16:12:04 -07:00
Archana Katiyar 40c1f072b4
HBASE-21207 Add client side sorting functionality in master web UI for table and region server details 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 15:29:49 -07:00
ramie-raufdeen 5fe40089f3
HBASE-19418 configurable range of delay in PeriodicMemstoreFlusher 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 14:27:28 -07:00
xcang c9c861dbb9
HBASE-18451 PeriodicMemstoreFlusher should inspect the queue before adding a delayed flush request, fix logging 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 11:50:19 -07:00
zhangduo 1109ebb617 HBASE-21244 Skip persistence when retrying for assignment related procedures 2018-09-28 21:09:41 +08:00
Allan Yang 7b71903baf HBASE-21228 Memory leak since AbstractFSWAL caches Thread object and never clean later 2018-09-27 16:11:47 +08:00
Michael Stack 38d6829a62 HBASE-21232 Show table state in Tables view on Master home page 2018-09-26 10:57:53 -07:00
zhangduo 51854a41fc HBASE-21227 Implement exponential retrying backoff for Assign/UnassignRegionHandler introduced in HBASE-21217 2018-09-26 20:54:06 +08:00
Allan Yang f3c3c7c422 HBASE-21212 Wrong flush time when update flush metric 2018-09-26 19:17:35 +08:00
Zach York 21fafbaf53 HBASE-20734 Colocate recovered edits directory with hbase.wal.dir 
Amending-Author: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-09-26 11:49:21 +08:00
Mingliang Liu 0f514ab75a
HBASE-21164 reportForDuty should do backoff rather than retry 
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.
 2018-09-25 11:32:15 -07:00
zhangduo fa2888ebf0 HBASE-21217 Revisit the executeProcedure method for open/close region 2018-09-25 18:10:47 +08:00
Andrew Purtell 20974513d9
Amend HBASE-20704 Sometimes some compacted storefiles are not archived on region close 
Forward port small logging improvements from branch-1 version of this change.
 2018-09-21 16:12:44 -07:00
Guangxu Cheng fd68e7593e
HBASE-20636 Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2018-09-21 16:06:34 -07:00
Michael Stack 29214c77a6
HBASE-21214 [hbck2] setTableState just sets hbase:meta state, not in-memory state 2018-09-21 16:04:42 -07:00
openinx 648d58bc23 HBASE-21206 Scan with batch size may return incomplete cells 2018-09-20 22:24:17 +08:00
tianjingyun 460cf6ff47 HBASE-21204 NPE when scan raw DELETE_FAMILY_VERSION and codec is not set 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-09-20 06:58:53 -07:00
Umesh Agashe 899fddb4e7
HBASE-21023 Added bypassProcedure() API to HbckService 2018-09-19 15:17:52 -07:00
Michael Stack 8876f12c0c
HBASE-21156 [hbck2] Queue an assign of hbase:meta and bulk assign/unassign 
Adds 'raw' assigns and unassigns methods to Hbck Service.

Fixes HbckService so it works when cluster is Kerberized.
 2018-09-19 09:47:32 -07:00
tedyu 84896deb1c HBASE-21102 ServerCrashProcedure should select target server where no other replicas exist for the current region - addendum fixes NPE 2018-09-19 07:17:55 -07:00
Vasudevan 13581e527d HBASE-21102 - ServerCrashProcedure should select target server where no 
other replicas exist for the current region -  fix flaky test (Ram)
 2018-09-18 11:30:49 +05:30
Michael Stack 6616fd2e0f HBASE-21191 Add a holding-pattern if no assign for meta or namespace (Can happen if masterprocwals have been cleared). 
Add a check for hbase:meta being online before we go to read it.
If not online, move into a holding-pattern until rectified, probably
by external operator.

Incorporates bulk of patch made by Allan Yang over on HBASE-21035.

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java

 Add a Constructor for case where retries are for ever.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Move stuff around so that the first hbase:meta read is the AM#loadMeta.
 Previously, checking table state and/or favored nodes could end up
 trying to read a meta that was not onlined holding up master startup.
 Do similar for the namespace table. Adds new methods isMeta and
 isNamespace which check that the regions/tables are online.. if not,
 we wait logging with a back-off that assigns need to be run.

Signed-off-by: Allan Yang <allan163@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-09-16 21:43:48 -07:00
Francis Liu 7a1b4d449d HBASE-20704 Sometimes some compacted storefiles are not archived on region close 2018-09-16 18:28:24 -07:00
Ted Yu 966e3751c2 HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning 
Amending-Author: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-09-15 18:36:33 +08:00
Mike Drob e9d23d5d25 HBASE-21168 Insecure Randomness in BloomFilterUtil 
Flagged by Fortify static analysis

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-09-12 09:51:57 -05:00
Tyler Mi d40348e8cd HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3 
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-09-11 16:31:56 -07:00
Nihal Jain c0acad92d2
HBASE-21021 Result returned by Append operation should be ordered 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-11 14:55:32 -07:00
Duo Zhang 6e699b0694 HBASE-21172 Addendum fix comment 2018-09-11 17:32:38 +08:00
Duo Zhang 55e1297d96 HBASE-21172 Reimplement the retry backoff logic for ReopenTableRegionsProcedure 2018-09-11 15:14:02 +08:00
Guangxu Cheng 52d50cc426 HBASE-21173 Remove the duplicate HRegion#close in TestHRegion 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-09-11 11:20:23 +08:00
Guangxu Cheng 1a7dba44ce HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter 2018-09-10 21:51:48 +08:00
Vasudevan 2be9196ba7 HBASE-21102 - ServerCrashProcedure should select target server where no 
other replicas exist for the current region (ram)
 2018-09-10 13:31:36 +05:30
Duo Zhang d35e2f35a4 HBASE-21144 AssignmentManager.waitForAssignment is not stable 2018-09-10 12:19:04 +08:00
Toshihiro Suzuki 49501cd5b5 HBASE-21052 After restoring a snapshot, table.jsp page for the table gets stuck 2018-09-09 10:25:10 +09:00
Guangxu Cheng 1b65c2f161 HBASE-21001 ReplicationObserver fails to load in HBase 2.0.0 2018-09-07 23:44:03 +08:00
Michael Stack 3f8c15ad2b
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir; ADDENDUM 
Address review comments.

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-09-06 17:25:20 -07:00
Michael Stack 62919a3791
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir 2018-09-06 16:45:53 -07:00
Mingliang Liu bbd9131c59 HBASE-21138 Close HRegion instance at the end of every test in TestHRegion 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-09-06 15:01:19 -07:00
Vasudevan 1798032ace HBASE-21161 Enable the test added in HBASE-20741 that was removed 
accidentally (Ram)
 2018-09-06 16:53:16 +05:30
Huaxiang Sun 83c871add8 HBASE-21107 add a metrics for netty direct memory 2018-09-05 16:10:17 -07:00
Guangxu Cheng dee13ea401 HBASE-20892 [UI] Start / End keys are empty on table.jsp 2018-09-05 09:21:38 +08:00
Vasudevan 1a5aedab34 HBASE-20741 Split of a region with replicas creates all daughter regions 
and its replica in same server (Ram)

Signed-off-by: Huaxiang Sun, Michael Stack, Duo Zhang
 2018-09-04 16:26:11 +05:30
Reid Chan 0409c54ba7 HBASE-21129 Clean up duplicate codes in #equals and #hashCode methods of Filter 
Signed-off-by Ted Yu <yuzhihong@gmail.com>
 2018-09-04 11:15:57 +08:00
David Manning 58365c8dd2 HBASE-21126 Configurable number of allowed failures for ZooKeeper Canary 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-09-03 19:35:28 -07:00
zhangduo 68c5313ca4 HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime 2018-09-02 18:32:24 +08:00
Duo Zhang cf59f2a1d2 HBASE-21128 TestAsyncRegionAdminApi.testAssignRegionAndUnassignRegion is broken 2018-08-30 10:09:49 +08:00
Xu Cang 72a29211cc HBASE-15728 Add Per-Table metrics back 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
	hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSource.java
	hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregate.java
	hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSourceImpl.java
	hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperStub.java
	hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableSourceImpl.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregateImpl.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableAggregate.java
 2018-08-29 16:27:15 -07:00
Allan Yang 737ac48473
HBASE-21083 Introduce a mechanism to bypass the execution of a stuck procedure 2018-08-28 20:19:55 -07:00
krish.dey 22254cf4ce HBASE-20942 Fix ArrayIndexOutOfBoundsException for RpcServer TRACE logging 
Also makes the trace log message length configurable.

Signed-off-by: Josh Elser <elserj@apache.org>
 2018-08-28 11:02:18 -04:00
zhangduo fc21dc854b HBASE-21017 Revisit the expected states for open/close 2018-08-28 14:49:47 +08:00
Umesh Agashe cdf5cfa396 HBASE-20941 Created and implemented HbckService in master 
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
 2018-08-27 12:10:33 -07:00
Mingliang Liu a0cbfb8ea6 HBASE-21071 HBaseTestingUtility::startMiniCluster() to use builder pattern 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-08-27 10:22:04 -07:00
tedyu f9790b01d9 HBASE-21088 HStoreFile should be closed in HStore#hasReferences 2018-08-27 09:38:03 -07:00
zhangduo 4f32883e95 HBASE-21072 Addendum do not write lock file when running TestHBaseFsckReplication 2018-08-27 21:04:48 +08:00
Toshihiro Suzuki 2369afd7e5 HBASE-21084 When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online 2018-08-27 09:58:52 +09:00
zhangduo 404d4d56cc HBASE-21095 The timeout retry logic for several procedures are broken after master restarts 2018-08-26 18:17:20 +08:00
zhangduo f533f01a3a HBASE-20881 Introduce a region transition procedure to handle all the state transition for a region 2018-08-26 18:08:06 +08:00
Michael Stack 2b7ef2e623 HBASE-20772 Controlled shutdown fills Master log with the disturbing message 'No matching procedure found for rit=OPEN, location=ZZZZ, table=YYYYY, region=XXXX transition to CLOSED' 
Look for the particular case where RS does the close of region w/o
involving Master and log special message in this case. Dodgy. But
until we have Master run shutdown of all regions, better than
the message we currently show.
 2018-08-25 14:30:27 +08:00
Michael Stack a83073aff0 HBASE-21078 [amv2] CODE-BUG NPE in RTP doing Unassign 2018-08-24 13:22:45 -07:00
Michael Stack 6047fb05f5 HBASE-21113 Apply the branch-2 version of HBASE-21095, The timeout retry logic for several procedures are broken after master restarts 
I applied the patch HBASE-21095 and then reverted it so could apply the
patch as HBASE-21113 (by reverting the HBASE-21095 revert but pushing
with this message!).

Revert "Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts""

This reverts commit a220566b98.
 2018-08-24 12:32:33 -07:00
Michael Stack a220566b98 Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts" 
This reverts commit 09be1d0d57.
 2018-08-24 12:24:17 -07:00
Allan Yang 09be1d0d57 HBASE-21095 The timeout retry logic for several procedures are broken after master restarts 2018-08-24 12:21:01 -07:00
Michael Stack 2ea45e67d0 HBASE-21072 Block out HBCK1 in hbase2 
Write the hbase-1.x hbck1 lock file to block out hbck1 instances writing
state to an hbase-2.x cluster (could do damage).
Set hbase.write.hbck1.lock.file to false to disable this writing.
 2018-08-24 09:23:54 -07:00
Duo Zhang c33af1e854 HBASE-21101 Remove the waitUntilAllRegionsAssigned call after split in TestTruncateTableProcedure 2018-08-24 10:35:05 +08:00
zhangduo b318311dfd HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file 2018-08-24 10:09:23 +08:00
tedyu 87f9b4accd HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning 2018-08-23 11:49:47 -07:00
liubangchen a7a281a644 HBASE-19008 Add missing equals or hashCode method(s) to stock Filter implementations 
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
 2018-08-24 00:39:35 +08:00
Duo Zhang cc6fc1f04f HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure 2018-08-23 06:27:37 +08:00
Allan Yang a7da458c5b HBASE-21041 Memstore's heap size will be decreased to minus zero after flush 2018-08-22 23:03:20 +08:00
Allan Yang 77e1a66f1e HBASE-21031 Memory leak if replay edits failed during region opening 2018-08-22 22:22:35 +08:00
Xu Cang b5ae4e5f9e HBASE-20666 Unsuccessful table creation leaves entry in hbase:rsgroup table 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-08-21 18:55:46 -07:00
Andrey Elenskiy d1b5113511 HBASE-21032 ScanResponses contain only one cell each 
Amending-Author: Duo Zhang <zhangduo@apache.org>
 2018-08-21 13:31:49 -07:00
tedyu 3787c72953 HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region - addendum 2018-08-19 14:25:12 -07:00
Andrew Purtell 52ba33aa73 HBASE-20940 HStore.cansplit should not allow split to happen if it has references (Vishal Khandelwal) 2018-08-17 15:02:22 -07:00
Andrew Purtell 1fa67725c5 HBASE-21047 Object creation of StoreFileScanner thru constructor and close may leave refCount to -1 (Vishal Khandelwal) 2018-08-16 11:42:49 -07:00
Sakthi 75939775af HBASE-20705 Having RPC quota on a table now no longer prevents Space Quota to be recreate/removed 
Just added 2 test cases as the subtasks of this jira solves the issue

Signed-off-by: Josh Elser <elserj@apache.org>
 2018-08-16 11:16:56 -04:00
Josh Elser cbe2fc113d HBASE-21062 Correctly use the defaultProvider value on the Providers enum when constructing a WALProvider 2018-08-16 10:18:39 -04:00
Nihal Jain d5abb92088 HBASE-20469 Directory used for sidelining old recovered edits files should be made configurable 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-08-15 18:01:24 -07:00
jingyuntian bee8566f2c HBASE-20985 add two attributes when we do normalization 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-13 16:49:37 +08:00
Duo Zhang d979f70142 HBASE-21025 Addendum missed a 'succ = true' 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-13 10:54:49 +08:00
brandboat 51df2887c4 HBASE-21012 Revert the change of serializing TimeRangeTracker 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-08-11 22:25:10 +08:00
zhangduo 2311fa86d4 HBASE-21025 Add cache for TableStateManager 2018-08-10 21:11:45 +08:00
brandboat fb9aa8a95a HBASE-18201 add UT and docs for DataBlockEncodingTool 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-08-10 11:16:36 +08:00
meiyi 28c3336bfe HBASE-20965 Separate region server report requests to new handlers 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-09 18:15:22 +08:00
Sakthi 6a8bd0fe84 HBASE-20813 Removed RPC quotas when the associated table/Namespace is dropped off 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-08-07 13:43:27 -04:00
jingyuntian 1aacde4839 HBASE-20986 Separate the config of block size when we do log splitting and write Hlog 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-07 14:00:31 +08:00
Sakthi b2cefab0e1 HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-08-03 11:06:56 -04:00
Monani Mihir 06a92a3d20 HBASE-19036 Add action in Chaos Monkey to restart Active Namenode 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-08-02 05:00:16 -07:00
TAK LON WU 690d29bae7 HBASE-20856 PITA having to set WAL provider in two places 
With this change if hbase.wal.meta_provider is not explicitly set,
it uses whatever set with hbase.wal.provider. this change avoids a use
case of unexpectedly using two different providers when only
hbase.wal.provider is set to non-default but not hbase.wal.meta_provider.

This change also include document (architecture.adoc) update

Also, this is a port from master to branch-2

Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <Apache9@apache.org>
 2018-08-01 14:36:29 -07:00
Michael Stack fc0c4660fa HBASE-20989 Minor, miscellaneous logging fixes 
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-08-01 11:20:26 -07:00
Xu Cang bfb9c71e3b HBASE-20794 add INFO level log to createTable operation 2018-08-01 11:04:18 -07:00
Michael Stack 477d567fd7 HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing ADDENDUM: Rather than rollback, just do region reopens. 
In split, reopen the parent if recovered.edits and in merge, reopen the
parent region or regions that happened to have recovered.edits on close.
 2018-08-01 00:33:33 -07:00
Andrew Purtell 777f3e88e2 HBASE-20935 HStore.removeCompactedFiles should log in case it is unable to delete a file (Vishal Khandelwal) 
Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
 2018-07-31 15:59:53 -07:00
Wei-Chiu Chuang 99714da30f HBASE-20950 Helper method to configure secure DFS cluster for tests 
Create a helper method HBaseKerberosUtils#setSecuredConfiguration().
TestSecureExport, TestSaslFanOutOneBlockAsyncDFSOutput,
SecureTestCluster and TestThriftSpnegoHttpServer uses this new helper
method.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-31 13:37:10 -07:00
Reid Chan e14b60a539 HBASE-20886 [Auth] Support keytab login in hbase client 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-07-31 11:10:34 +08:00
zhangduo 584093c23f HBASE-20538 Upgrade our hadoop versions to 2.7.7 and 3.0.3 2018-07-29 20:20:01 +08:00
zhangduo 5957fceda4 Revert "HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3" 
This reverts commit 71a4c17d99.
 2018-07-29 20:19:16 +08:00
zhangduo 71a4c17d99 HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3 2018-07-29 20:04:53 +08:00
Alex Leblang ab18440107
HBASE-19369 Switch to Builder Pattern In WAL 
This patch switches to the builder pattern by adding a helper method.
It also checks to ensure that the pattern is available (i.e. that
HBase is running on a hadoop version that supports it).

Amending-Author: Mike Drob <mdrob@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-27 23:42:53 -05:00
zhangduo 51b0d0edd0 HBASE-20939 There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException 2018-07-27 21:25:22 +08:00
Allan Yang 611ca11c9f HBASE-20921 Possible NPE in ReopenTableRegionsProcedure 2018-07-27 09:34:25 +08:00
Mingliang Liu bb6f24cd59 HBASE-20932 Effective MemStoreSize::hashCode() 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-25 14:42:47 -07:00
Peter Somogyi 3b84c9c8d2 HBASE-20649 Validate HFiles do not have PREFIX_TREE DataBlockEncoding 
Amending-Author: Balazs Meszaros <balazs.meszaros@cloudera.com>

Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-07-25 09:12:17 -05:00
Allan Yang ac4b789f5f HBASE-20867 RS may get killed while master restarts 2018-07-25 18:13:50 +08:00
zhangduo 0b283dae0d HBASE-20846 Restore procedure locks when master restarts 2018-07-25 14:37:31 +08:00
Saurabh Singh 097ae55708 HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-24 08:17:16 -07:00
huzheng 99701968c8 HBASE-20565 ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result 2018-07-24 10:33:33 +08:00
Allan Yang aaf5c78445 HBASE-20878 Data loss if merging regions while ServerCrashProcedure executing 2018-07-24 09:57:47 +08:00
Allan Yang 786fd9137a HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing 2018-07-23 14:46:32 +08:00
tedyu 654480326b HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region 2018-07-22 20:51:58 -07:00
Reid Chan dd0325a740 HBASE-20401 Make MAX_WAIT and waitIfNotFinished in CleanerContext configurable (Contributed by Stephen Wu) 2018-07-23 10:32:17 +08:00
Toshihiro Suzuki b15efbf8ed HBASE-19893 restore_snapshot is broken in master branch when region splits 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-21 07:47:50 -07:00
Ankit Singhal e8a7a4472f HBASE-20908 Infinite loop on regionserver if region replica are reduced 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-20 13:05:43 -07:00
Michael Stack 810473b277 HBASE-20914 Trim Master memory usage 
Add (weak reference) interning of ServerNames.

Correct Balancer regions x racks matrix.

Make smaller defaults when creating ArrayDeques.
 2018-07-20 10:08:37 -07:00
Allan Yang 1a6fae74b5 HBASE-20870 Wrong HBase root dir in ITBLL's Search Tool 2018-07-20 11:22:03 +08:00
Ankit 37e5f5fb5f HBASE-20672 New metrics ReadRequestRate and WriteRequestRate 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-07-19 15:41:53 -07:00
Ankit Singhal cf1f63bf91 HBASE-20901 Reducing region replica has no effect 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-19 13:39:44 -07:00
Mohit Goel 35a3c605f2 HBASE-6028 Start/Stop compactions at region server level 
Add switching on/off of compactions.

Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.

Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-07-19 06:20:12 -07:00
Yu Li e464230bde HBASE-20907 Fix Intermittent failure on TestProcedurePriority 2018-07-19 12:01:16 +08:00
Michael Stack 532d84d6d0 HBASE-20875 MemStoreLABImp::copyIntoCell uses 7% CPU when writing 
Make the #copyCellInto method smaller so it inlines; we do it by
checking for the common type early and then taking a code path
that presumes ByteBufferExtendedCell -- avoids checks.
 2018-07-18 20:41:05 -07:00
Allan Yang 3078acb83c HBASE-20860 Merged region's RIT state may not be cleaned after master restart 2018-07-17 17:33:11 +08:00
tedyu 07fcdf9418 HBASE-20879 Compacting memstore config should handle lower case 2018-07-13 11:22:06 -07:00
Toshihiro Suzuki d3bb364595 HBASE-20865 CreateTableProcedure is stuck in retry loop in CREATE_TABLE_WRITE_FS_LAYOUT state 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-13 08:26:05 -07:00
Toshihiro Suzuki d7f4b06206 HBASE-19572 RegionMover should use the configured default port number and not the one from HConstants 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-13 10:46:01 +08:00
zhangduo 59830b0078 HBASE-20847 Addendum use addFront instead of addBack to add sub procedure 2018-07-12 08:31:21 +08:00
zhangduo 430b056ade HBASE-20847 The parent procedure of RegionTransitionProcedure may not have the table lock 2018-07-11 17:37:21 +08:00
eshcar 176afd998a HBASE-20542-ADDENDUM: fix TestHStore 2018-07-11 11:31:28 +03:00
zhaoyuan 7990283d50 HBASE-20697 Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations() 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-07-11 11:07:04 +08:00
zhangduo 8decae9b92 HBASE-20784 Will lose the SNAPSHOT suffix if we get the version of RS from ServerManager 2018-07-10 10:00:11 +08:00
Balazs Meszaros e6403c18ff
HBASE-20833 Modify pre-upgrade coprocessor validator to support table level coprocessors 
- -jar parameter now accepts multiple jar files and directories of jar files.
- observer classes can be verified by -class option.
- -table parameter was added to check table level coprocessors.
- -config parameter was added to obtain the coprocessor classes from
  HBase cofiguration.
- -scan option was removed.

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-07-09 11:15:18 -05:00
Abhishek Singh Chouhan ae22d57201 HBASE-20806 Split style journal for flushes and compactions 2018-07-09 18:42:30 +05:30
zhangduo 8a394ade4c HBASE-20822 TestAsyncNonMetaRegionLocator is flakey 2018-07-09 14:56:42 +08:00
Nihal Jain 1da2fa3baa HBASE-20808 (Addendum) Remove duplicate calls for cancelling of chores 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-07 00:18:22 +08:00
Nihal Jain 473c71edb1 HBASE-20808 Wrong shutdown order between Chores and ChoreService 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-06 11:37:13 +08:00
Yu Li 4653d4ac6b HBASE-20691 Change the default WAL storage policy back to "NONE"" 
This reverts commit 564c193d61 and added more doc
about why we choose "NONE" as the default.
 2018-07-04 13:45:16 +08:00
Guangxu Cheng 341fa55367 HBASE-20474 Show non-RPC tasks on master/regionserver Web UI by default 2018-07-04 10:56:46 +08:00
zhangduo 5ef9d8f451 HBASE-20839 Fallback to FSHLog if we can not instantiated AsyncFSWAL when user does not specify AsyncFSWAL explicitly 2018-07-04 10:29:30 +08:00
zhangduo c612642b2b HBASE-20829 Remove the addFront assertion in MasterProcedureScheduler.doAdd 2018-07-04 09:40:52 +08:00
Ted Yu 021eae48ca HBASE-20244 NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 22:15:48 +08:00
huzheng 8b1e6a81ff HBASE-20789 TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky 2018-07-03 17:59:25 +08:00
jingyuntian 30664b1a57 HBASE-20193 Basic Replication Web UI - Regionserver 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 16:02:28 +08:00
Josh Elser 33d002f445 HBASE-20826 Truncate really long RpcServer warnings unless TRACE is on 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-07-03 10:14:52 +08:00
Nihal Jain ad7e453817 HBASE-20450 Provide metrics for number of total active, priority and replication rpc handlers 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-02 11:38:02 -07:00
Ankit Singhal 3847def1c2 HBASE-20817 Infinite loop when executing ReopenTableRegionsProcedure 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-02 21:28:53 +08:00
eshcar 7868f0ef97 HBASE-20542: Better heap utilization for IMC with MSLABs 2018-07-01 15:52:30 +03:00
Ankit Singhal 490ca0ce20 HBASE-20825 Fix pre and post hooks of CloneSnapshot and RestoreSnapshot for Access checks 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
 2018-06-29 16:33:34 -04:00
Pankaj a16e5523fc HBASE-20357 AccessControlClient API Enhancement 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-06-29 07:54:14 -07:00
Josh Elser 02235f4e32 HBASE-20792 info:servername and info:sn inconsistent for OPEN region 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-06-29 11:11:09 +08:00
Xu Cang 58ccd3dc7e HBASE-19722 Meta query statistics metrics source 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-06-28 17:17:20 -07:00
Michael Stack f264b79ec1 HBASE-20781 Save recalculating families in a WALEdit batch of Cells 
Pass the Set of families through to the WAL rather than recalculate
a Set already known.

Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-27 22:04:40 -07:00
Reid Chan 32e0767dfd HBASE-20732 Shutdown scan pool when master is stopped 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-28 11:51:52 +08:00
Sahil Aggarwal e61507b9a0
HBASE-19164: Remove UUID.randomUUID in tests. 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-27 10:36:48 -05:00
jingyuntian 557985ee47 HBASE-20194 Basic Replication WebUI - Master 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-26 18:27:50 +08:00
zhangduo a1b297e673 HBASE-20777 RpcConnection could still remain opened after we shutdown the NettyRpcServer 2018-06-26 09:07:58 +08:00
Michael Stack 40cd76af8a HBASE-20780 ServerRpcConnection logging cleanup Get rid of one of the logging lines in ServerRpcConnection by amalgamating all into one new-style log line. 2018-06-25 16:43:43 -07:00
Michael Stack 4e16ecd7f4
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do 
General log cleanup; setting stuff that can flood the log to TRACE.
 2018-06-25 12:12:43 -07:00
Todd Lipcon bf65e95466 HBASE-20403. Fix race between prefetch task and non-pread HFile reads 
With prefetch-on-open enabled, the task doing the prefetching was using
non-positional (i.e. streaming) reads. If the main (non-prefetch) thread
was also using non-positional reads, these two would conflict, because
inputstreams are not thread-safe for non-positional reads.

In the case of an encrypted filesystem, this could cause JVM crashes,
etc, as underlying cipher buffers were freed underneath the racing
threads. In the case of a non-encrypted filesystem, less severe errors
would be thrown. The included unit test reproduces the latter case.

(cherry picked from commit 025ddce868)
Signed-off-by: Todd Lipcon <todd@cloudera.com>
 2018-06-25 12:11:36 -07:00
Michael Stack e915024e3c HBASE-20778 Make it so WALPE runs on DFS 2018-06-23 23:34:17 -07:00
zhangduo 2f4b0a4a42 HBASE-20775 TestMultiParallel is flakey 2018-06-24 08:26:26 +08:00
zhangduo dde042cc93 HBASE-20776 Update branch-2 version to 2.2.0-SNAPSHOT 2018-06-22 22:15:18 +08:00
zhangduo 108e636247 HBASE-18569 Add prefetch support for async region locator 2018-06-22 18:20:08 +08:00
zhangduo a86141b625 HBASE-20752 Make sure the regions are truly reopened after ReopenTableRegionsProcedure 2018-06-22 14:06:29 +08:00
zhangduo 6cebe06225 HBASE-20767 Always close hbaseAdmin along with connection in HBTU 2018-06-22 10:20:06 +08:00
Ankit Singhal 28d0d8c5cc HBASE-20642 Clients should re-use the same nonce across DDL operations 
Also changes modify table operations to help the case where a MTP spans
two master, avoiding the sanity-checks propagating back to the client
unnecessarily.

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-06-20 15:10:52 -07:00
Josh Elser c8b76eb3f1 HBASE-20706 Prevent MTP from trying to reopen non-OPEN regions 
ModifyTableProcedure is using MoveRegionProcedure in a way
that was unintended from the original implementation. As such,
we have to guard against certain usages of it. We know we can
re-open OPEN regions, but regions in OPENING will similarly
soon be OPEN (thus, we want to reopen those regions too).

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-20 14:30:36 -07:00
zhangduo 5c2cb15e0b HBASE-20739 Add priority for SCP 2018-06-20 15:18:47 +08:00
zhangduo 83969b0da1 HBASE-20742 Always create WAL directory for region server 2018-06-20 14:21:28 +08:00
Michael Stack 9eeb501825 HBASE-20745 Log when master proc wal rolls 2018-06-19 19:53:29 -07:00
zhangduo 3e33aecea2 HBASE-20708 Remove the usage of RecoverMetaProcedure in master startup 2018-06-19 15:09:11 +08:00
Sean Busbey ee84a8f243 HBASE-20332 shaded mapreduce module shouldn't include hadoop 
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports

 Conflicts:
	hbase-backup/pom.xml
	hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-18 14:02:48 -07:00
Mike Drob b04c976fe6 HBASE-20478 Update checkstyle to v8.2 
Cannot go to latest (8.9) yet due to
  https://github.com/checkstyle/checkstyle/issues/5279

* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors

Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-06-18 14:02:40 -07:00
taiynlee 8edd5d948a HBASE-20737 put collection into ArrayList instead of addAll function 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-17 11:16:16 +08:00
tedyu b2afba580b HBASE-20723 Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them 2018-06-16 01:34:53 -07:00
Xu Cang b68746c0b2 HBASE-20695 Implement table level RegionServer replication metrics 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-06-15 10:45:13 +08:00
jingyuntian bde9f08a83 HBASE-20625 refactor some WALCellCodec related code 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-06-14 19:46:33 +08:00
zhangduo 161dc7c7f3 HBASE-20722 Make RegionServerTracker only depend on children changed event 2018-06-14 08:38:53 +08:00
Guanghao Zhang 075523dd1e HBASE-20561 The way we stop a ReplicationSource may cause the RS down 2018-06-13 18:05:27 +08:00
Balazs Meszaros d44e8a7aff HBASE-20656 Validate pre-2.0 coprocessors against HBase 2.0+ 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-11 10:32:40 -05:00
Mike Drob 4b0bbd839e HBASE-20707 Move MissingSwitchDefault case check 
Perform this check using error-prone instead of checkstyle because the
former can handle enum switches somewhat more intelligently.
 2018-06-11 10:13:29 -05:00
zhangduo 6befdc43ba HBASE-20700 Move meta region when server crash can cause the procedure to be stuck 2018-06-11 15:28:21 +08:00
Guanghao Zhang 4d971d0f48 HBASE-20698 (addendum) Master don't record right server version until new started region server call regionServerReport method 2018-06-10 08:32:01 +08:00
Guanghao Zhang 9d15e16946 HBASE-20698 Master don't record right server version until new started region server call regionServerReport method 2018-06-09 14:47:07 +08:00
Nihal Jain 4a5fe54d94 HBASE-20699 QuotaCache should cancel the QuotaRefresherChore service inside its stop() 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-06-08 07:22:52 -07:00
Michael Stack 858eee20ec HBASE-20702 Processing crash, skip ONLINE'ing empty rows 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-06-07 09:54:32 -07:00
eric-maynard 271d93dc73 HBASE-20665: Changed log level of HBASE-8547 warning to debug 
Closes #77

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-06-07 11:34:52 -04:00
Peter Somogyi 00289b8ffa HBASE-20683 Incorrect return value for PreUpgradeValidator 
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-06 20:04:22 +02:00
Andrew Purtell d7b09de854 HBASE-20670 NPE in HMaster#isInMaintenanceMode 2018-06-04 15:19:45 -07:00
Michael Stack 063eefe3b0 HBASE-20634 Reopen region while server crash can cause the procedure to be stuck; ADDENDUM 2018-06-04 12:38:56 -07:00
Michael Stack 27e2c8c86b HBASE-20628 SegmentScanner does over-comparing when one flushing 
Signed-off-by: eshcar <eshcar@oath.com>
Signed-off-by: anoopsjohn <anoopsamjohn@gmail.com>
 2018-06-04 09:50:13 -07:00
zhangduo d834859404 HBASE-20634 Reopen region while server crash can cause the procedure to be stuck 
A reattempt at fixing HBASE-20173 [AMv2] DisableTableProcedure concurrent to ServerCrashProcedure can deadlock

The scenario is a SCP after processing WALs, goes to assign regions that
were on the crashed server but a concurrent Procedure gets in there
first and tries to unassign a region that was on the crashed server
(could be part of a move procedure or a disable table, etc.). The
unassign happens to run AFTER SCP has released all RPCs that
were going against the crashed server. The unassign fails because the
server is crashed. The unassign used to suspend itself only it would
never be woken up because the server it was going against had already
been processed. Worse, the SCP could not make progress because the
unassign was suspended with the lock on a region that it wanted to
assign held making it so it could make no progress.

In here, we add to the unassign recognition of the state where it is
running post SCP cleanup of RPCs. If present, unassign moves to finish
instead of suspending itself.

Includes a nice unit test made by Duo Zhang that reproduces nicely the
hung scenario.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/FailedRemoteDispatchException.java
 Moved this class back to hbase-procedure where it belongs.

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NoNodeDispatchException.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NoServerDispatchException.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NullTargetServerDispatchException.java
 Specializiations on FRDE so we can be more particular when we say there
 was a problem.

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
 Change addOperationToNode so we throw exceptions that give more detail
 on issue rather than a mysterious true/false

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Undo SERVER_CRASH_HANDLE_RIT2. Bad idea (from HBASE-20173)

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 Have expireServer return true if it actually queued an expiration. Used
 later in this patch.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Hide methods that shouldn't be public. Add a particular check used out
 in unassign procedure failure processing.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
 Check that server we're to move from is actually online (might
 catch a few silly move requests early).

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
 Add doc on ServerState. Wasn't being used really. Now we actually stamp
 a Server OFFLINE after its WAL has been split. Means its safe to assign
 since all WALs have been processed. Add methods to update SPLITTING
 and to set it to OFFLINE after splitting done.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 Change logging to be new-style and less repetitive of info.
 Cater to new way in which .addOperationToNode returns info (exceptions
 rather than true/false).

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 Add looking for the case where we failed assign AND we should not
 suspend because we will never be woken up because SCP is beyond
 doing this for all stuck RPCs.

 Some cleanup of the failure processing grouping where we can proceed.

 TODOs have been handled in this refactor including the TODO that
 wonders if it possible that there are concurrent fails coming in
 (Yes).

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
 Doc and removing the old HBASE-20173 'fix'.
 Also updating ServerStateNode post WAL splitting so it gets marked
 OFFLINE.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestServerCrashProcedureStuck.java
 Nice test by Duo Zhang.

Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Duo Zhang <palomino219@gmail.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-04 09:26:36 -07:00
maoling 4c95b82b61 HBASE-19761:Fix Checkstyle errors in hbase-zookeeper 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2018-06-02 10:17:27 +02:00
Andrew Purtell f46569a742 HBASE-20667 Rename TestGlobalThrottler to TestReplicationGlobalThrottler 2018-06-01 17:01:14 -07:00
Xu Cang d3e2248f12 HBASE-18116 Replication source in-memory accounting should not include bulk transfer hfiles 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-06-01 11:16:16 -07:00
Peter Somogyi 53d29d53c4 HBASE-20592 Create a tool to verify tables do not have prefix tree encoding 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-01 19:22:49 +02:00
Andrew Purtell b22409d51d Revert "HBASE-18116 fix replication source in-memory calculation by excluding bulk load file" 
This reverts commit 050fae501a.
 2018-05-31 15:28:37 -07:00
Xu Cang 050fae501a HBASE-18116 fix replication source in-memory calculation by excluding bulk load file 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-31 14:22:12 -07:00
Sean Busbey fc9743c17a HBASE-20444 Addendum keep folks from looking at raw version component array. 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-31 14:17:41 -05:00
Andrew Purtell aaec02e0f5 HBASE-20646 TestWALProcedureStoreOnHDFS failing on branch-1 2018-05-30 14:44:54 -07:00
Andrew Purtell 15bb234d51 Revert "TestWALProcedureStoreOnHDFS failing on branch-1" 
This reverts commit 694e79a67e.
 2018-05-30 14:44:49 -07:00
Andrew Purtell 694e79a67e TestWALProcedureStoreOnHDFS failing on branch-1 2018-05-30 13:46:08 -07:00
zhangduo b785896cbd HBASE-20659 Implement a reopen table regions procedure 2018-05-30 20:03:35 +08:00
tedyu 856a3ac154 HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint - revert due to pending discussion 2018-05-29 19:58:32 -07:00
Andrew Purtell 2dc51934f4 HBASE-20597 Serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint 2018-05-29 11:29:12 -07:00
Andrew Purtell 7f154dc20e Revert "HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint" 
This reverts commit 60dcef289b.
 2018-05-29 11:24:30 -07:00
Nihal Jain d36cce1574 HBASE-20633 Dropping a table containing a disable violation policy fails to remove the quota upon table delete 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-05-29 11:50:40 -04:00
eshcar aa00391140 HBASE-20390 ADDENDUM 2: fix TestHRegionWithInMemoryFlush OOME 2018-05-29 16:24:27 +03:00
eshcar cf1928aaca HBASE-20390-ADDENDUM: fix TestHRegionWithInMemoryFlush OOME 2018-05-29 13:01:07 +03:00
huzheng c8fd6e0fb6 HBASE-20533 Fix the flaky TestAssignmentManagerMetrics 2018-05-29 09:50:04 +08:00
Toshihiro Suzuki 0455e75edd HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-28 08:27:41 -07:00
Nihal Jain bc72fcd8c5 HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-27 18:43:08 -07:00
meiyi f40c10a211 HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-05-25 14:45:49 +08:00
Thiruvel Thirumoolan d1cbd561df HBASE-20548 Master fails to startup on large clusters, refreshing block distribution 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-24 15:47:22 -07:00
Toshihiro Suzuki db8789ab22 HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-24 15:17:01 -07:00
Nihal Jain 55c4791a63 HBASE-20588 Space quota change after quota violation doesn't seem to take in effect 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-05-24 12:50:22 -04:00
eshcar 079e08d7c5 HBASE-20390: IMC Default Parameters for 2.0.0 2018-05-24 18:10:10 +03:00
Guanghao Zhang bfab1e2f92 HBASE-20589 Don't need to assign meta to a new RS when standby master become active 2018-05-24 11:45:59 +08:00
zhangduo a94c6dbadb HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper 2018-05-24 10:48:35 +08:00
Andrew Purtell 60dcef289b HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint 2018-05-23 16:46:20 -07:00
Michael Stack afddf6b1c2 HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2 
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.

Remove unused and unneeded Interfaces from Cell base type.
 2018-05-23 13:20:29 -07:00
huzheng fbda502435 HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn 2018-05-23 12:07:54 +08:00
jingyuntian c3c9a4a595 HBASE-20579 Improve snapshot manifest copy in ExportSnapshot 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-18 06:43:53 -07:00
Guanghao Zhang d06673cf3e HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal 2018-05-18 14:30:40 +08:00
Balazs Meszaros 39ea1efa88 HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics 
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
  "NaN", "Infinity" or "-Infinity"

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-16 12:20:37 -07:00
Apekshit Sharma 61f2b5f071 HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace. 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-05-16 14:03:36 -05:00
Michael Stack 77eaff0e10
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM 
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.

Includes a nothing change to hbase-server/pom.xml just to provoke
build.
 2018-05-16 09:42:51 -07:00
Michael Stack 5c4685e56e
HBASE-20520 Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize 2018-05-16 09:18:06 -07:00
zhangduo 82e3011166 HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler 2018-05-16 08:46:34 +08:00
zhangduo 60b8344cf1 HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream 2018-05-15 21:09:04 +08:00
Zach York b7def9b690 HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata 
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-14 16:09:14 -07:00
huzheng 4b0ac73f51 HBASE-20560 Revisit the TestReplicationDroppedTables ut 2018-05-14 19:33:51 +08:00
huzheng be3df29cef HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL 2018-05-14 19:32:39 +08:00
Michael Stack 5ac7740896
HBASE-20411 Ameliorate MutableSegment synchronize 
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:

 + Make MemStoreSize, a datastructure of our memstore size longs, immutable.
 + Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
 + Make two MemStoreSizing implementations; one thread-safe, the other not.
 + Let all memory sizing longs run independent, untied by
   synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
 + Review all use of MemStoreSizing. Many are single-threaded and do
   not need to be synchronized; use the non-thread safe counter.

TODO: Use this technique accounting at the global level too.
 2018-05-12 02:16:19 +01:00
Thiruvel Thirumoolan 1f10ef553e HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-10 10:45:17 -07:00
Sean Busbey 61f96b6ffa HBASE-20544 Make HBTU default to random ports. 
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>

 Conflicts:
	hbase-backup/src/test/resources/hbase-site.xml
	hbase-spark-it/src/test/resources/hbase-site.xml
	hbase-spark/src/test/resources/hbase-site.xml
 2018-05-09 23:45:39 -07:00
Andrew Purtell c430016cf9 HBASE-20554 "WALs outstanding" message from CleanerChore is noisy 2018-05-09 19:11:50 -07:00
Zach York cba8d2fb8d HBASE-20204 Add locking to RefreshFileConnections in BucketCache 
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
 2018-05-09 14:23:27 -07:00
Michael Stack de25f8b209 HBASE-20539 Disable IMC; part 2 
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
 2018-05-09 06:29:56 -07:00
Nihal Jain 55badefa38 HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-05-09 16:45:39 +08:00
Guanghao Zhang d6d209564c HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number 2018-05-08 08:21:30 +08:00
Michael Stack fd0840c3e4 HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property 
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
 2018-05-07 09:54:07 -07:00
huzheng de71cb5915 HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication 2018-05-04 15:52:43 +08:00
jingyuntian 5f260451d5 HBASE-20378 Provide a hbck option to cleanup replication barrier for a table 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-05-04 15:20:17 +08:00
Guanghao Zhang eaafdff761 HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources 2018-05-04 14:34:07 +08:00
Michael Stack e92580e201 HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer 2018-05-03 12:26:54 -07:00
tedyu f89c5b08df HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine 2018-05-03 07:12:11 -07:00
Chia-Ping Tsai 984fb5bd05
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT 2018-05-02 16:14:38 -07:00