Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,465 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

7403 Commits

Author SHA1 Message Date
huzheng 81ebe6d40d HBASE-21764 Size of in-memory compaction thread pool shoud be configurable 2019-02-01 10:01:34 +08:00
Nihal Jain c5f4e84106 HBASE-21644 Modify table procedure runs infinitely for a table having region replication > 1 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-01-31 19:34:22 +08:00
Nihal Jain 1c71544497 HBASE-21404 Master/RS navbar active state does not work 
Signed-off-by: Jingyun Tian <tianjy@apache.org>
 2019-01-31 16:33:07 +08:00
Sergey Shelukhin c90e9ff5ef HBASE-21806 add an option to roll WAL on very slow syncs 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-01-30 16:43:42 -08:00
meiyi 5ddda1a1f6 HBASE-21733 SnapshotQuotaObserverChore should only fetch space quotas 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-30 17:51:25 +08:00
wanghuan 16c7f5dac9 HBASE-21699 Fixed create table failed when using SPLITS_FILE => 'splits.txt' 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-30 15:56:25 +08:00
meiyi f997252344 HBASE-21739 Move grant/revoke from regionserver to master 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-30 11:16:57 +08:00
Duo Zhang 7dc69b6128 HBASE-21792 Mark HTableMultiplexer as deprecated and remove it in 3.0.0 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-29 09:08:02 +08:00
zhangduo 274e4ccea8 HBASE-21789 Rewrite MetaTableAccessor.multiMutate with Table.coprocessorService 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-27 11:36:58 +08:00
zhangduo 1995f61d7f HBASE-21782 LoadIncrementalHFiles should not be IA.Public 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-27 11:24:55 +08:00
Duo Zhang 4f3b95b318 HBASE-21762 Move some methods in ClusterConnection to Connection 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-01-25 21:13:44 +08:00
meiyi 281558a3ab HBASE-21713 Support set region server throttle quota 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-25 13:50:42 +08:00
Wellington Chevreuil 54093b0d1f HBASE-20215 - Rename CollectionUtils to ConcurrentMapUtils 2019-01-24 20:55:15 -08:00
Jingyun Tian 91dffb043a HBASE-21729 Extract ProcedureCoordinatorRpcs and ProcedureMemberRpcs from CoordinatedStateManager 2019-01-25 11:15:55 +08:00
zhangduo e43c833145 HBASE-21770 Should deal with meta table in HRegionLocator.getAllRegionLocations 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-25 11:00:51 +08:00
zhangduo 416b70f461 HBASE-21761 Align the methods in RegionLocator and AsyncTableRegionLocator 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-24 15:21:11 +08:00
Tommy Li d187af00ca HBASE-21720 : metric to measure how actions are distributed to servers within a MultiAction 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2019-01-23 17:52:58 -08:00
Nihal Jain 050caf425e
HBASE-21475 Put mutation (having TTL set) added via co-processor is retrieved even after TTL expires 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-01-23 16:57:28 -08:00
Allan Yang b5619a2a26 HBASE-21754 ReportRegionStateTransitionRequest should be executed in priority executor 2019-01-23 20:55:00 +08:00
Duo Zhang dfad304ddb HBASE-21753 Support getting the locations for all the replicas of a region 2019-01-23 18:09:39 +08:00
Sergey Shelukhin fa3946fbea HBASE-21626 log the regions blocking WAL from being archived ADDENDUM 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-22 15:15:38 -08:00
huzheng 35ed5d6c39 HBASE-21750 Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized 2019-01-22 19:58:03 +08:00
Nihal Jain f2820ea16f
HBASE-21749 RS UI may throw NPE and make rs-status page inaccessible with multiwal and replication 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-01-21 17:26:07 -08:00
huzheng 35df6147ee HBASE-21738 Remove all the CLSM#size operation in our memstore because it's an quite time consuming. 2019-01-21 16:52:49 +08:00
zhangduo a2f6768acd HBASE-21746 Fix two concern cases in RegionMover 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-20 08:57:01 +08:00
Duo Zhang 882bd564f4 HBASE-21726 Add getAllRegionLocations method to AsyncTableRegionLocator 2019-01-19 11:14:35 +08:00
zhangduo 44dc872b7b HBASE-19695 Handle disabled table for async client 
Signed-off-by: tianjingyun <tianjy1990@gmail.com>
 2019-01-17 18:34:12 +08:00
Beluga Behr 3e92cd5771 HBASE-20209 Do Not Use Both Map containsKey and get Methods in Replication Sink 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2019-01-16 09:39:40 -07:00
Duo Zhang 594341d6fe HBASE-21710 Add quota related methods to the Admin interface 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-01-16 21:08:47 +08:00
huzheng 15031c5bd5 HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case 2019-01-16 09:41:02 +08:00
tianjingyun c2c1af4927 HBASE-21647 Add status track for splitting WAL tasks 2019-01-15 16:38:37 -08:00
Pankaj 6da0b4ec34 HBASE-21639 maxHeapUsage value not read properly from config during EntryBuffers initialization 2019-01-15 12:18:28 -08:00
Pankaj bc0fb66e22 HBASE-21595, Print thread's information and stack traces when RS is aborting forcibly 2019-01-15 12:11:48 -08:00
Sakthi 93d4b95b3f HBASE-21225: Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE' 
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-01-14 17:55:42 -07:00
Sergey Shelukhin e69ab24552 HBASE-21626 : log the regions blocking WAL from being archived 2019-01-14 14:52:24 -08:00
zhangduo c5691a9f60 HBASE-21705 Should treat meta table specially for some methods in AsyncAdmin 2019-01-13 13:40:52 +08:00
Duo Zhang 3d2580cd6d HBASE-21704 The implementation of DistributedHBaseCluster.getServerHoldingRegion is incorrect 2019-01-11 21:34:16 +08:00
Duo Zhang fbf79373e6 HBASE-21663 Add replica scan support 2019-01-11 14:19:05 +08:00
Duo Zhang 7bebdff6a2 HBASE-21580 Support getting Hbck instance from AsyncConnection 2019-01-11 11:26:44 +08:00
Nihal Jain 5d32e80f9e HBASE-21297 ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-10 22:01:45 +08:00
Guanghao Zhang 8e39ec2c59 HBASE-21691 Fix flaky test TestRecoveredEdits 2019-01-10 10:06:45 +08:00
meiyi 620d70d618 HBASE-21694 Add append_peer_exclude_tableCFs and remove_peer_exclude_tableCFs shell commands 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-09 22:02:21 +08:00
Guanghao Zhang a5a61163f1 HBASE-21695 Fix flaky test TestRegionServerAbortTimeout 2019-01-09 21:58:19 +08:00
Sergey Shelukhin 37bc1686d6 HBASE-21614 RIT recovery with ServerCrashProcedure doesn't account for all regions 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2019-01-09 20:03:37 +08:00
Nihal Jain 5c902b48e5 HBASE-21645 Perform sanity check and disallow table creation/modification with region replication < 1 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-09 10:49:19 +08:00
Vladimir Rodionov d26acbe1db HBASE-21688: Address WAL filesystem issues 
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-01-08 13:56:11 -05:00
zhangduo e9dae2adc3 HBASE-21683 Reset readsEnabled flag after successfully flushing the primary region 2019-01-08 16:06:34 +08:00
zhangduo 4f0514e39a HBASE-21682 Support getting from specific replica 2019-01-08 09:49:12 +08:00
tianjingyun 281d6429e5 HBASE-21588 Procedure v2 wal splitting implementation 2019-01-07 15:58:15 +08:00
meiyi 77db1fae09 HBASE-21159 Add shell command to switch throttle on or off 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-01-07 14:21:41 +08:00
zhangduo db66e6cc9e HBASE-17356 Add replica get support 2019-01-03 08:38:20 +08:00
Guanghao Zhang ec948f5d90 HBASE-21659 Avoid to load duplicate coprocessors in system config and table descriptor 2019-01-02 10:41:36 +08:00
Guanghao Zhang 3ab895979b HBASE-21660 Apply the cell to right memstore for increment/append operation 2019-01-01 17:32:44 +08:00
Guanghao Zhang f5ea00f724 HBASE-21643 Introduce two new region coprocessor method and deprecated postMutationBeforeWAL 2018-12-27 18:27:06 +08:00
huzheng c2d5991b82 HBASE-21642 CopyTable by reading snapshot and bulkloading will save a lot of time 2018-12-27 18:22:54 +08:00
Guanghao Zhang 4281cb3b95 HBASE-21640 Remove the TODO when increment zero 2018-12-26 21:47:44 +08:00
Andrey Elenskiy dbafa1be83 HBASE-21545 NEW_VERSION_BEHAVIOR breaks Get/Scan with specified columns 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sakthi
 2018-12-23 22:01:11 -08:00
zhangduo 97fd647de2 HBASE-21635 Use maven enforcer to ban imports from illegal packages 2018-12-24 11:12:25 +08:00
Guanghao Zhang 7c0a3cc265 HBASE-21621 Reversed scan does not return expected number of rows 
The unit test is contributed by Nihal Jain
 2018-12-23 16:19:05 +08:00
Sakthi b2bf22e209 HBASE-21631: list_quotas should print human readable values for LIMIT 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-12-22 22:00:58 +08:00
openinx e160b5ac8d HBASE-21620 Problem in scan query when using more than one column prefix filter in some cases 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-12-21 15:21:53 -08:00
huzheng 12786f80c1 HBASE-21401 Sanity check when constructing the KeyValue 2018-12-21 18:01:35 +08:00
Guanghao Zhang ad819380c7 HBASE-21618 Scan with the same startRow(inclusive=true) and stopRow(inclusive=false) returns one result 2018-12-21 09:49:24 +08:00
Pankaj 787567336a HBASE-21610, numOpenConnections metric is set to -1 when zero server channel exist 2018-12-20 16:36:42 -08:00
Guanghao Zhang 8991877bb2 HBASE-21514: Refactor CacheConfig(addendum) 2018-12-19 13:55:13 +08:00
Pankaj fb58a23e56 HBASE-21535, Zombie Master detector is not working 2018-12-18 20:51:01 -08:00
Jingyun Tian c448604ceb HBASE-21565 Delete dead server from dead server list too early leads to concurrent Server Crash Procedures(SCP) for a same server 2018-12-18 16:57:11 +08:00
xuqinya f78284685f HBASE-21592 quota.addGetResult(r) throw NPE 
Signed-off-by: huzheng <openinx@gmail.com>
 2018-12-18 16:15:51 +08:00
Guanghao Zhang 1971d02e72 HBASE-21514 Refactor CacheConfig 2018-12-18 13:43:30 +08:00
stack 68b5df0095 HBASE-21589 TestCleanupMetaWAL fails 2018-12-17 09:31:59 -08:00
huzheng ac0b3bb547 HBASE-21520 TestMultiColumnScanner cost long time when using ROWCOL bloom type 2018-12-15 21:08:52 +08:00
Sean Busbey 491153488e HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. (addendum) 2018-12-14 17:08:22 -06:00
meiyi 1b08ba7385 HBASE-21578 Fix wrong throttling exception for capacity unit 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-12-14 18:17:47 +08:00
Sergey Shelukhin 3ff274e22e HBASE-21575 : memstore above high watermark message is logged too much 2018-12-13 12:47:11 -08:00
Sergey Shelukhin 9a25d0c249 Revert "HIVE-21575 : memstore above high watermark message is logged too much" 
This reverts commit 4640ff5959.
 2018-12-13 12:46:39 -08:00
Lars Hofhansl cb1966dc2d HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. 2018-12-13 11:57:16 -08:00
huzheng f32d261843 HBASE-21582 If call HBaseAdmin#snapshotAsync but forget call isSnapshotFinished, then SnapshotHFileCleaner will skip to run every time 2018-12-13 10:35:20 +08:00
Sergey Shelukhin 4640ff5959 HIVE-21575 : memstore above high watermark message is logged too much 2018-12-12 11:02:25 -08:00
Josh Elser 67d6d5084c HBASE-21568 Use CacheConfig.DISABLED where we don't expect to have blockcache running 
This includes removing the "old way" of disabling blockcache in favor of the
new API.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-12-11 10:02:18 -05:00
zhangduo b09b87d143 HBASE-21570 Add write buffer periodic flush support for AsyncBufferedMutator 2018-12-11 14:51:26 +08:00
Guanghao Zhang 79d90c87b5 HBASE-21560 Return a new TableDescriptor for MasterObserver#preModifyTable to allow coprocessor modify the TableDescriptor 2018-12-08 09:28:14 +08:00
Guanghao Zhang 8d7061a487 HBASE-21554 Show replication endpoint classname for replication peer on master web UI 2018-12-07 13:31:59 +08:00
Allan Yang e0e0694fa1 HBASE-21413 Empty meta log doesn't get split when restart whole cluster 
Signed-off-by: stack <stack@apache.org>
 2018-12-06 21:21:41 -08:00
huzheng dfb9ae8e0e HBASE-21559 The RestoreSnapshotFromClientTestBase related UT are flaky 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-12-07 08:22:22 +08:00
Tommy Li 170df27b88 HBASE-21414 : StoreFileSize growth rate metric 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2018-12-06 14:43:17 -08:00
huzheng 67ab8b888f HBASE-21551 Memory leak when use scan with STREAM at server side - (addendum) 2018-12-06 11:26:52 +08:00
huzheng 3b854859f6 HBASE-21551 Memory leak when use scan with STREAM at server side 2018-12-06 10:55:42 +08:00
Duo Zhang f49baf259e HBASE-21550 Add a new method preCreateTableRegionInfos for MasterObserver which allows CPs to modify the TableDescriptor 2018-12-06 08:30:32 +08:00
zhangduo 8bf966c8e9 HBASE-21541 Move MetaTableLocator.verifyRegionLocation to hbase-rsgroup module 2018-12-04 22:33:36 +08:00
zhangduo dec49135a3 HBASE-21534 Addendum output uncaught exception, fix the way that we add new region servers to avoid recreate a dead server 2018-12-03 08:19:21 +08:00
Duo Zhang d525ec6a12 HBASE-21534 TestAssignmentManager is flakey 2018-12-01 20:41:11 +08:00
Peter Somogyi 5c209f2cec HBASE-21479 Individual tests in TestHRegionReplayEvents class are failing 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-12-01 09:36:10 +01:00
Duo Zhang 766aa1bfcc HBASE-21486 The current replication implementation for peer in STANDBY state breaks serial replication 2018-12-01 12:15:18 +08:00
Peter Somogyi dfeab9f5c9 HBASE-21518 TestMasterFailoverWithProcedures is flaky 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-11-30 09:12:06 +01:00
huzheng d42e0ade1c HBASE-21504 If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived. 2018-11-30 09:53:06 +08:00
Artem Ervits 3ad2a89fdf HBASE-18735 Provide an option to kill a MiniHBaseCluster without waiting on shutdown 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-11-29 16:37:50 -05:00
meiyi f1f2b5a038 HBASE-21300 Fix the wrong reference file path when restoring snapshots for tables with MOB columns 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-29 11:23:45 +08:00
Duo Zhang d6e1d18be9 HBASE-21517 Move the getTableRegionForRow method from HMaster to TestMaster 2018-11-28 09:40:27 +08:00
BELUGA BEHR 7877e09b60 HBASE-21492 CellCodec Written To WAL Before It's Verified 2018-11-27 08:57:06 -08:00
lixiaobao 6f15cecaed HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
 2018-11-27 20:56:23 +08:00
zhangduo 39bc458eab Revert "HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method" 
Email address is broken.

This reverts commit 3133d43859.
 2018-11-27 20:55:04 +08:00
lixiaobao 3133d43859 HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
 2018-11-27 20:31:07 +08:00
Guanghao Zhang 34e1d744f7 HBASE-21498 Master OOM when SplitTableRegionProcedure new CacheConfig and instantiate a new BlockCache 2018-11-27 14:20:15 +08:00
zhangduo 1acbd36c90 HBASE-21154 Remove hbase:namespace table; fold it into hbase:meta 2018-11-26 14:51:56 +08:00
Duo Zhang a0e3cb6c0c HBASE-21508 Ignore the reportRegionStateTransition call from a dead server 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-26 14:32:14 +08:00
Ted Yu 27c0bf5c63 HBASE-21511 Remove in progress snapshot check in SnapshotFileCache#getUnreferencedFiles 2018-11-25 18:20:00 -08:00
zhangduo 701526d19f HBASE-21510 Test TestRegisterPeerWorkerWhenRestarting is flakey 2018-11-24 19:50:59 +08:00
Ted Yu d9c773b0a5 HBASE-21387 Addendum fix TestSnapshotFileCache 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-11-24 17:00:08 +08:00
huzheng 6d0dc960e6 HBASE-21387 Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files 2018-11-23 11:31:37 +08:00
Duo Zhang 5cc845b713 HBASE-21503 Replication normal source can get stuck due potential race conditions between source wal reader and wal provider initialization threads. 
Found and analysed by Wellington Chevreuil
 2018-11-21 17:14:54 +08:00
meiyi 5ded294419 HBASE-21034 Add new throttle type: read/write capacity unit 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-21 09:46:49 +08:00
Duo Zhang 405bf5e638 HBASE-21490 WALProcedure may remove proc wal files still with active procedures 
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-11-19 08:21:28 -08:00
zhangduo b329e6e3f2 HBASE-21494 NPE when loading RecoverStandByProcedure 2018-11-19 09:35:18 +08:00
zhangduo f555258e7a HBASE-21480 Taking snapshot when RS crashes prevent we bring the regions online 2018-11-18 21:07:22 +08:00
zhangduo 43a10df70f HBASE-21485 Add more debug logs for remote procedure execution 2018-11-16 14:30:59 +08:00
Reid Chan 130057f137 HBASE-21255 [acl] Refactor TablePermission into three classes (Global, Namespace, Table) 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-11-15 11:34:16 +08:00
zhangduo 9e42a9e310 HBASE-21472 Should not persist the dispatched field for RegionRemoteProcedureBase 2018-11-15 09:41:38 +08:00
huzheng a8b642067d HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags 2018-11-14 23:00:12 +08:00
zhangduo 640a5e390b HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors 2018-11-14 18:48:33 +08:00
zhangduo 64c4861272 Revert "HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors" 
This reverts commit ffb003ee44.
 2018-11-13 22:24:59 +08:00
zhangduo 3eb27a099a Revert "HBASE-21465 Addendum change TestReportRegionStateTransitionRetry to MediumTests" 
This reverts commit 114377d426.
 2018-11-13 22:24:50 +08:00
Duo Zhang 114377d426 HBASE-21465 Addendum change TestReportRegionStateTransitionRetry to MediumTests 2018-11-13 14:59:20 +08:00
Duo Zhang ffb003ee44 HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors 2018-11-13 11:49:22 +08:00
Duo Zhang 55fa8f4b33 HBASE-21463 The checkOnlineRegionsReport can accidentally complete a TRSP 2018-11-13 11:31:03 +08:00
tedyu f770081129 HBASE-21466 WALProcedureStore uses wrong FileSystem if wal.dir is not under rootdir 2018-11-12 17:02:17 -08:00
huzheng 362b5dd259 Revert "HBASE-21401 Sanity check in BaseDecoder#parseCell" 
This reverts commit f17382792f.
 2018-11-12 11:37:46 +08:00
Duo Zhang 6786aca666 HBASE-21376 Add some verbose log to MasterProcedureScheduler 2018-11-10 17:17:12 +08:00
Duo Zhang fe2265fa4a HBASE-21451 The way we maintain the lastestPaths in ReplicationSourceManager is broken when sync replication is used 2018-11-09 14:53:33 +08:00
jingyuntian fa6373660f HBASE-21410 A helper page that help find all problematic regions and procedures 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-11-09 14:01:49 +08:00
Guanghao Zhang 13b4351012 Revert "HBASE-21410 A helper page that help find all problematic regions and procedures" 
This reverts commit 3c34073988.

missing sign-off.
 2018-11-09 14:01:29 +08:00
jingyuntian 3c34073988 HBASE-21410 A helper page that help find all problematic regions and procedures 2018-11-09 13:53:00 +08:00
Ben Lau d9f32137b6 HBASE-21439 RegionLoads aren't being used in RegionLoad cost functions 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-11-08 11:59:34 -08:00
Esteban Gutierrez a8ad61ec88
HBASE-20604 ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-11-08 10:22:22 -08:00
huzheng f17382792f HBASE-21401 Sanity check in BaseDecoder#parseCell 2018-11-08 20:07:04 +08:00
jingyuntian cf9b515563 HBASE-21419 Show sync replication related field for replication peer on master web UI 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-11-08 14:50:37 +08:00
zhangduo 6d46b8d256 HBASE-21441 NPE if RS restarts between REFRESH_PEER_SYNC_REPLICATION_STATE_ON_RS_BEGIN and TRANSIT_PEER_NEW_SYNC_REPLICATION_STATE 2018-11-07 16:10:42 +08:00
Allan Yang a31458bdee HBASE-21421 Do not kill RS if reportOnlineRegions fails 2018-11-06 14:58:35 +08:00
tedyu 7395ffac44 HBASE-21247 Custom WAL Provider cannot be specified by configuration whose value is outside the enums in Providers 2018-11-05 08:29:04 -08:00
Duo Zhang c8574ba3c5 HBASE-21420 Use procedure event to wake up the SyncReplicationReplayWALProcedures which wait for worker 2018-11-05 21:43:18 +08:00
Michael Stack d08990c962 HBASE-21425 2.1.1 fails to start over 1.x data; namespace not assigned 2018-11-03 09:46:19 -07:00
zhangduo 62fe365934 HBASE-21351 The force update thread may have race with PE worker when the procedure is rolling back 2018-11-03 08:24:11 +08:00
jingyuntian 25c964e9a3
HBASE-21407 Resolve NPE in backup Master UI 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-11-02 11:47:02 -07:00
zhangduo e7f6c2972d HBASE-21422 NPE in TestMergeTableRegionsProcedure.testMergeWithoutPONR 2018-11-02 20:54:00 +08:00
Beluga Behr 21b98e1075 HBASE-19682 Use Collections.emptyList() For Empty List Values 2018-11-01 08:51:11 -07:00
Guanghao Zhang 6bdaedd7ce HBASE-21388 No need to instantiate MemStoreLAB for master which not carry table 2018-11-01 16:15:18 +08:00
zhangduo 02f5f171f5 HBASE-21389 Revisit the procedure lock for sync replication 2018-10-30 18:11:20 +08:00
tianjingyun 116eee6747
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-29 20:55:37 -07:00
zhangduo d5e4faacc3 HBASE-21375 Revisit the lock and queue implementation in MasterProcedureScheduler 2018-10-29 19:56:49 +08:00
Guanghao Zhang c0b994b0c6 HBASE-21325 Force to terminate regionserver when abort hang in somewhere 2018-10-29 10:08:51 +08:00
Artem Ervits 7cdb525192 HBASE-21175 Partially initialized SnapshotHFileCleaner leads to NPE during TestHFileArchiving 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-10-26 16:44:33 -07:00
Duo Zhang 0ab7c3a189 HBASE-21391 RefreshPeerProcedure should also wait master initialized before executing 2018-10-26 21:44:56 +08:00
Guanghao Zhang 23b7510ae2 HBASE-21365 Throw exception when user put data with skip wal to a table which may be replicated 2018-10-26 17:43:38 +08:00
Guanghao Zhang cd943419b6 HBASE-21385 HTable.delete request use rpc call directly instead of AsyncProcess 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-25 16:22:36 -07:00
Michael Stack 6830a1c1a0 HBASE-21372) Set hbase.assignment.maximum.attempts to Long.MAX 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-10-24 09:07:40 -07:00
Duo Zhang 3fe8649b2c HBASE-21377 Add debug log for catching the root cause 2018-10-24 15:43:12 +08:00
xcang 1f437ac221
HBASE-21349 Do not run CatalogJanitor or Nomalizer when cluster is shutting down 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-23 14:45:05 -07:00
mazhenlin 1e9d998727
HBASE-21342 FileSystem in use may get closed by other bulk load call in secure bulkLoad 
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
 2018-10-23 15:51:46 -05:00
xcang 807736fcf1
HBASE-21338 Warn if balancer is an ill-fit for cluster size 
Signed-off-by: Michael Stack <stack@apache.org>
 2018-10-23 13:23:24 -07:00
huzheng ae13b0b293 HBASE-21356 bulkLoadHFile API should ensure that rs has the source hfile's write permissionls 2018-10-23 10:18:40 +08:00
zhangduo 931156f66b HBASE-21336 Addendum remove unused code in HBTU 2018-10-22 20:26:48 +08:00
huzheng d0e7367c8d HBASE-21355 (addendum) replace the expensive reload storefiles with reading the merge result of compacted storefiles and current storefiles 2018-10-22 19:28:53 +08:00
zhangduo 3b66b65b9f HBASE-21336 Simplify the implementation of WALProcedureMap 2018-10-22 18:36:11 +08:00
huzheng 77ac352a95 HBASE-21355 HStore's storeSize is calculated repeatedly which causing the confusing region split 2018-10-22 09:43:14 +08:00
Duo Zhang 7d7293049a Revert "HBASE-21336 Simplify the implementation of WALProcedureMap" 
This reverts commit 7adf590106.
 2018-10-22 09:32:55 +08:00
zhangduo dd474ef199 HBASE-21334 TestMergeTableRegionsProcedure is flakey 2018-10-21 22:40:19 +08:00
Artem Ervits b723ce1051 HBASE-21194 Add tests in TestCopyTable which exercises MOB feature 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-10-20 08:10:27 -07:00
zhangduo 7adf590106 HBASE-21336 Simplify the implementation of WALProcedureMap 2018-10-20 21:59:46 +08:00
Toshihiro Suzuki 4bf3c5a702 HBASE-21200 Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner. 2018-10-20 08:34:23 +09:00
Mike Drob bc7628a8c3
HBASE-21073 Redo concept of maintenance mode 
Instead of being an ephemeral state set by hbck, maintenance mode is now
an explicit toggle set by either configuration property or environment
variable. In maintenance mode, master will host system tables and not
assign any user-space tables to RSs. This gives operators the ability to
affect repairs to meta table with fewer moving parts.
 2018-10-18 16:27:53 -05:00
jingyuntian 5fbb227deb
HBASE-21269　Forward-port HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign 2018-10-18 06:22:52 -07:00
haxiaolin 92fdc8dd51 HBASE-21055 NullPointerException when balanceOverall() but server balance info is null 
Signed-off-by: huzheng <openinx@gmail.com>
 2018-10-18 14:04:28 +08:00
Duo Zhang 5efa5f6de4 HBASE-21330 ReopenTableRegionsProcedure will enter an infinite loop if we schedule a TRSP at the same time 2018-10-18 11:29:12 +08:00
Michael Stack fd940f3b78
HBASE-21327 Fix minor logging issue where we don't report servername if no associated SCP 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-10-17 09:42:45 -07:00
Michael Stack 8cc56bd18c
HBASE-21320 [canary] Cleanup of usage and add commentary 
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
 2018-10-16 22:20:00 -07:00
subrat.mishra 3b91ae5b2a
HBASE-21263 Mention compression algorithm along with other storefile details 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2018-10-16 12:38:11 -07:00
zhangduo 0d9982901a HBASE-21278 Do not rollback successful sub procedures when rolling back a procedure 2018-10-16 15:12:49 +08:00
Andrew Purtell c9dcc9a065
HBASE-21266 Not running balancer because processing dead regionservers, but empty dead rs list 2018-10-15 22:28:36 -07:00
zhangduo fc7a6a64c2 HBASE-21311 Split TestRestoreSnapshotFromClient 2018-10-16 08:29:35 +08:00
Guanghao Zhang 4a043126b1 HBASE-21290 No need to instantiate BlockCache for master which not carry table 2018-10-15 17:21:17 +08:00
haxiaolin 7d798b3c73 HBASE-21260 The whole balancer plans might be aborted if there are more than one plans to move a same region 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-10-15 15:54:13 +08:00
zhangduo 6781918ca8 HBASE-21310 Split TestCloneSnapshotFromClient 2018-10-15 14:00:40 +08:00
zhangduo dde336f6ef HBASE-21309 Increase the waiting timeout for TestProcedurePriority 2018-10-14 08:46:18 +08:00
Michael Stack a292ab7c87 HBASE-21299 List counts of actual region states in master UI tables section 2018-10-12 11:02:36 -07:00
Guanghao Zhang fa5fa6ecdd HBASE-21289 Remove the log "'hbase.regionserver.maxlogs' was deprecated." in AbstractFSWAL 2018-10-12 20:27:10 +08:00
Duo Zhang 9e9a1e0f0d HBASE-21254 Need to find a way to limit the number of proc wal files 2018-10-12 11:05:13 +08:00
tedyu 924d183ba0 HBASE-21247 Allow WAL Provider to be specified by configuration without explicit enum in Providers - revert 2018-10-11 09:00:24 -07:00
Mike Drob db9a5b7da7 HBASE-21287 Allow configuring test master initialization wait time. 2018-10-11 09:43:04 -05:00
tedyu 0789f54454 HBASE-21247 Allow WAL Provider to be specified by configuration without explicit enum in Providers 2018-10-10 08:27:16 -07:00
Guanghao Zhang a1f28f3ca7 HBASE-21277 Prevent to add same table to two sync replication peer's config 2018-10-10 15:43:16 +08:00
Guanghao Zhang f122328758 HBASE-21251 Refactor RegionMover 2018-10-10 15:10:57 +08:00
Michael Stack 7c755bf06e HBASE-21280 Add anchors for each heading in UI 
Signed-off-by: Ted Yu <tedyu@apache.org>
 2018-10-09 22:50:19 -07:00
Duo Zhang e8df847d4e HBASE-21250 Addendum remove unused modification in hbase-server module 2018-10-08 14:55:55 +08:00
zhangduo 118b074684 HBASE-21250 Refactor WALProcedureStore and add more comments for better understanding the implementation 2018-10-07 17:09:09 +08:00
Wellington Chevreuil 4508f670b1 HBASE-21185 - WALPrettyPrinter: Additional useful info to be printed by wal printer tool, for debugability purposes 
Signed-off-by: Allan Yang <allan163@apache.org>
 2018-10-04 03:32:18 -07:00
tedyu b9bb14e0fd HBASE-21221 Ineffective assertion in TestFromClientSide3#testMultiRowMutations - addendum tightens condition for finding exception 2018-10-03 08:22:57 -07:00
Xu Cang 42aa3dd463
HBASE-18549 Add metrics for failed replication queue recovery 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-10-01 18:38:55 -07:00
Xu Cang 1b7e4fdcfc
HBASE-19275 TestSnapshotFileCache never worked properly 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-10-01 17:12:00 -07:00
Guanghao Zhang f9d51b67e4 HBASE-21245 Add exponential backoff when retrying for sync replication related procedures 2018-09-30 20:05:34 +08:00
zhangduo fdbaa4c3f0 HBASE-21248 Implement exponential backoff when retrying for ModifyPeerProcedure 2018-09-29 13:19:36 +08:00
zhangduo ab6ec1f9e4 Revert "HBASE-21248 Implement exponential backoff when retrying for ModifyPeerProcedure" 
This reverts commit 3baafbed52.
 2018-09-29 09:27:51 +08:00
Nihal Jain 56ac4705e9
HBASE-21196 HTableMultiplexer clears the meta cache after every put operation 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 16:35:46 -07:00
Kiran Kumar Maturi aa9e1d0511
HBASE-20857 balancer status tag in jmx metrics 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 16:12:15 -07:00
Archana Katiyar 801fc05e9c
HBASE-21207 Add client side sorting functionality in master web UI for table and region server details 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 15:28:20 -07:00
ramie-raufdeen 668a179882
HBASE-19418 configurable range of delay in PeriodicMemstoreFlusher 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 14:27:07 -07:00
xcang 704f8b81bb
HBASE-18451 PeriodicMemstoreFlusher should inspect the queue before adding a delayed flush request, fix logging 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-28 11:50:08 -07:00
zhangduo d39ea25d44 HBASE-21244 Skip persistence when retrying for assignment related procedures 2018-09-28 21:22:23 +08:00
zhangduo 3baafbed52 HBASE-21248 Implement exponential backoff when retrying for ModifyPeerProcedure 2018-09-28 20:20:48 +08:00
Allan Yang 86cb8e48ad HBASE-21228 Memory leak since AbstractFSWAL caches Thread object and never clean later 2018-09-27 16:24:17 +08:00
Michael Stack 1154f81230 HBASE-21232 Show table state in Tables view on Master home page 2018-09-26 10:58:16 -07:00
zhangduo d7e08317d2 HBASE-21227 Implement exponential retrying backoff for Assign/UnassignRegionHandler introduced in HBASE-21217 2018-09-26 20:53:56 +08:00
Allan Yang 7b2f5950ed HBASE-21212 Wrong flush time when update flush metric 2018-09-26 19:38:15 +08:00
Zach York 0e173d38b0 HBASE-20734 Colocate recovered edits directory with hbase.wal.dir 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-09-26 11:20:57 +08:00
Mingliang Liu 08c4d70aaf
HBASE-21164 reportForDuty should do backoff rather than retry 
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.
 2018-09-25 11:32:28 -07:00
tedyu b8134fe5b6 HBASE-21221 Ineffective assertion in TestFromClientSide3#testMultiRowMutations 2018-09-25 09:17:26 -07:00
zhangduo 8eaaa63114 HBASE-21217 Revisit the executeProcedure method for open/close region 2018-09-25 17:30:21 +08:00
Andrew Purtell 7ab77518a2
Amend HBASE-20704 Sometimes some compacted storefiles are not archived on region close 
Forward port small logging improvements from branch-1 version of this change.
 2018-09-21 16:12:33 -07:00
Guangxu Cheng 3de02d57f6
HBASE-20636 Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
 2018-09-21 16:06:39 -07:00
Michael Stack 98909f4a9d
HBASE-21214 [hbck2] setTableState just sets hbase:meta state, not in-memory state 2018-09-21 16:04:59 -07:00
openinx ddd30a2241 HBASE-21206 Scan with batch size may return incomplete cells 2018-09-20 22:36:24 +08:00
tianjingyun cd161d976e HBASE-21204 NPE when scan raw DELETE_FAMILY_VERSION and codec is not set 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-09-20 06:57:50 -07:00
Umesh Agashe dc767c06d2
HBASE-21023 Added bypassProcedure() API to HbckService 2018-09-19 15:18:16 -07:00
Michael Stack 3a0fcd56cf
HBASE-21156 [hbck2] Queue an assign of hbase:meta and bulk assign/unassign 
Adds 'raw' assigns and unassigns methods to Hbck Service.

Fixes HbckService so it works when cluster is Kerberized.
 2018-09-19 10:06:16 -07:00
tedyu 69431c75c1 HBASE-21102 ServerCrashProcedure should select target server where no other replicas exist for the current region - addendum fixes NPE 2018-09-19 07:17:13 -07:00
Vasudevan cebb725a9f HBASE-21102 - ServerCrashProcedure should select target server where no 
other replicas exist for the current region -  fix flaky test (Ram)
 2018-09-18 11:28:23 +05:30
tedyu 1cf920db43 HBASE-21160 Assertion in TestVisibilityLabelsWithDeletes#testDeleteColumnsWithoutAndWithVisibilityLabels is ignored (liubangchen) 2018-09-17 08:25:11 -07:00
Michael Stack 6d7bc0e98b HBASE-21191 Add a holding-pattern if no assign for meta or namespace (Can happen if masterprocwals have been cleared). 
Add a check for hbase:meta being online before we go to read it.
If not online, move into a holding-pattern until rectified, probably
by external operator.

Incorporates bulk of patch made by Allan Yang over on HBASE-21035.

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java

 Add a Constructor for case where retries are for ever.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Move stuff around so that the first hbase:meta read is the AM#loadMeta.
 Previously, checking table state and/or favored nodes could end up
 trying to read a meta that was not onlined holding up master startup.
 Do similar for the namespace table. Adds new methods isMeta and
 isNamespace which check that the regions/tables are online.. if not,
 we wait logging with a back-off that assigns need to be run.

Signed-off-by: Allan Yang <allan163@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-09-16 21:44:17 -07:00
Francis Liu cdfe808892 HBASE-20704 Sometimes some compacted storefiles are not archived on region close 2018-09-16 18:26:28 -07:00
Ted Yu 9da20a4116 HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning 
Amending-Author: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2018-09-15 18:20:35 +08:00
Mike Drob 0075093d21
HBASE-21168 Insecure Randomness in BloomFilterUtil 
Flagged by Fortify static analysis

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-09-12 09:51:45 -05:00
Tyler Mi 5c1b325b51 HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3 
Signed-off-by: Zach York <zyork@apache.org>
 2018-09-11 16:23:16 -07:00
Nihal Jain c372175da4
HBASE-21021 Result returned by Append operation should be ordered 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-09-11 14:43:52 -07:00
Duo Zhang 2ab8122a24 HBASE-21172 Addendum fix comment 2018-09-11 17:32:01 +08:00
Duo Zhang c59ecfb961 HBASE-21172 Reimplement the retry backoff logic for ReopenTableRegionsProcedure 2018-09-11 15:13:58 +08:00
Guangxu Cheng bea26e98e6 HBASE-21173 Remove the duplicate HRegion#close in TestHRegion 
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-09-11 11:06:23 +08:00
Guangxu Cheng 2aae247e3f HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter 2018-09-10 21:32:46 +08:00
Vasudevan b09dbb443e HBASE-21102 - ServerCrashProcedure should select target server where no 
other replicas exist for the current region (fix compilation issue)
 2018-09-10 13:23:53 +05:30
Vasudevan dc3ada2614 HBASE-21102 ServerCrashProcedure should select target server where no 
other replicas exist for the current region (Ram)
 2018-09-10 13:12:02 +05:30
Toshihiro Suzuki a51c333856 HBASE-21052 After restoring a snapshot, table.jsp page for the table gets stuck 2018-09-09 10:23:50 +09:00
zhangduo b04b4b0fd1 HBASE-21144 Addendum fix race when testing whether a procedure is finished 2018-09-08 08:57:56 +08:00
Guangxu Cheng c3419be003 HBASE-21001 ReplicationObserver fails to load in HBase 2.0.0 2018-09-07 23:42:02 +08:00
Michael Stack aab3a8a870
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir; ADDENDUM 
Address review comments.

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-09-06 17:25:39 -07:00
Michael Stack 3ac3249423
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir 2018-09-06 16:52:47 -07:00
Vasudevan b3908a26ff HBASE-21161 Enable the test added in HBASE-20741 that was removed 
accidentally (Ram)
 2018-09-06 16:51:40 +05:30
Mingliang Liu a37c40faa5 HBASE-21138 Close HRegion instance at the end of every test in TestHRegion 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-09-06 03:59:44 -07:00
Huaxiang Sun bdc168713d HBASE-21107 add a metrics for netty direct memory 2018-09-05 16:04:25 -07:00
Guangxu Cheng b62ac3cb11 HBASE-20892 [UI] Start / End keys are empty on table.jsp 2018-09-05 09:09:04 +08:00
Vasudevan c6a65ba63f HBASE-20741 - Split of a region with replicas creates all daughter regions 
and its replica in same server (Addendum for duo's comments in RB)
 2018-09-04 16:38:09 +05:30
Duo Zhang f504c4d797 HBASE-21144 AssignmentManager.waitForAssignment is not stable 2018-09-04 17:07:51 +08:00
Vasudevan 83131b1ac4 HBASE-20741 Split of a region with replicas creates all daughter regions 
and its replica in same server (Ram)

Signed-off-by: Huaxiang Sun, Michael Stack
 2018-09-04 11:18:51 +05:30
Vasudevan e4c4035ed8 HBASE-20741 Split of a region with replicas creates all daughter regions 
and its replica in same server (Ram)

Signed-off-by: Huaxiang Sun, Michael Stack
 2018-09-04 11:08:03 +05:30