Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,628 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

6985 Commits

Author SHA1 Message Date
Yu Li 619e0fdab8 HBASE-20907 Fix Intermittent failure on TestProcedurePriority 2018-07-19 12:00:55 +08:00
Michael Stack 2bf5e46a33 HBASE-20875 MemStoreLABImp::copyIntoCell uses 7% CPU when writing 
Make the #copyCellInto method smaller so it inlines; we do it by
checking for the common type early and then taking a code path
that presumes ByteBufferExtendedCell -- avoids checks.
 2018-07-18 20:41:19 -07:00
tedyu 2997b6d071 HBASE-20879 Compacting memstore config should handle lower case 2018-07-13 11:19:45 -07:00
Toshihiro Suzuki ce82fd0f47 HBASE-19572 RegionMover should use the configured default port number and not the one from HConstants 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-13 10:44:35 +08:00
Toshihiro Suzuki 3fc23fe930 HBASE-20865 CreateTableProcedure is stuck in retry loop in CREATE_TABLE_WRITE_FS_LAYOUT state 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-12 00:15:26 -07:00
Allan Yang 51527f551b HBASE-20860 Merged region's RIT state may not be cleaned after master restart 2018-07-12 10:12:07 +08:00
zhangduo 4a08135dd8 HBASE-20847 Addendum use addFront instead of addBack to add sub procedure 2018-07-12 08:31:05 +08:00
zhangduo a838f7631f HBASE-20847 The parent procedure of RegionTransitionProcedure may not have the table lock 2018-07-11 17:34:35 +08:00
eshcar 1804b6d059 HBASE-20542-ADDENDUM: fix TestHStore 2018-07-11 11:25:49 +03:00
zhaoyuan 1e0650955a HBASE-20697 Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations() 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-07-11 10:52:06 +08:00
zhangduo d7561cee50 HBASE-20784 Will lose the SNAPSHOT suffix if we get the version of RS from ServerManager 2018-07-10 09:59:41 +08:00
Balazs Meszaros ad5b4af2c4
HBASE-20833 Modify pre-upgrade coprocessor validator to support table level coprocessors 
- -jar parameter now accepts multiple jar files and directories of jar files.
- observer classes can be verified by -class option.
- -table parameter was added to check table level coprocessors.
- -config parameter was added to obtain the coprocessor classes from
  HBase cofiguration.
- -scan option was removed.

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-07-09 11:00:36 -05:00
Abhishek Singh Chouhan 59867eeeeb HBASE-20806 Split style journal for flushes and compactions 2018-07-09 18:25:34 +05:30
zhangduo b5222f88b2 HBASE-20822 TestAsyncNonMetaRegionLocator is flakey 2018-07-09 14:56:37 +08:00
Guanghao Zhang 3bca01854a HBASE-20842 Infinite loop when replaying remote wals 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-08 09:35:45 +08:00
Nihal Jain 361be53344 HBASE-20808 (Addendum) Remove duplicate calls for cancelling of chores 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-07 00:17:10 +08:00
Nihal Jain 1ade4d2f44 HBASE-20808 Wrong shutdown order between Chores and ChoreService 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-06 11:35:03 +08:00
Yu Li ec8947f226 HBASE-20691 Change the default WAL storage policy back to "NONE"" 
This reverts commit 564c193d61 and added more doc
about why we choose "NONE" as the default.
 2018-07-04 13:43:48 +08:00
Guangxu Cheng ee3990e42c HBASE-20474 Show non-RPC tasks on master/regionserver Web UI by default 2018-07-04 10:53:02 +08:00
zhangduo 4366720bd1 HBASE-20839 Fallback to FSHLog if we can not instantiated AsyncFSWAL when user does not specify AsyncFSWAL explicitly 2018-07-04 10:29:24 +08:00
Ted Yu 0f23784182 HBASE-20244 NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 22:15:18 +08:00
huzheng 0454878e71 HBASE-20789 TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky 2018-07-03 17:56:34 +08:00
jingyuntian 66ad9fdef8 HBASE-20193 Basic Replication Web UI - Regionserver 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 15:47:14 +08:00
zhangduo 380350d5bc HBASE-20829 Remove the addFront assertion in MasterProcedureScheduler.doAdd 2018-07-03 15:43:20 +08:00
Josh Elser 13e4578be8 HBASE-20826 Truncate really long RpcServer warnings unless TRACE is on 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-07-03 10:14:34 +08:00
Ankit Singhal cfdabe9267 HBASE-20817 Infinite loop when executing ReopenTableRegionsProcedure 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-02 21:26:14 +08:00
eshcar d822ee3a7c HBASE-20542: Better heap utilization for IMC with MSLABs 2018-07-01 15:31:31 +03:00
zhangduo 112d050609 HBASE-20829 TestSyncReplicationStandbyKillRS is flakey - add error log for debugging 2018-07-01 18:14:10 +08:00
Ankit Singhal 34e23fe425 HBASE-20825 Fix pre and post hooks of CloneSnapshot and RestoreSnapshot for Access checks 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
 2018-06-29 16:33:02 -04:00
Pankaj bb8826ca5f HBASE-20357 AccessControlClient API Enhancement 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-06-28 22:48:58 -07:00
Josh Elser fe75f90be2 HBASE-20792 info:servername and info:sn inconsistent for OPEN region 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-06-29 11:10:40 +08:00
Xu Cang 78e7dd6537 HBASE-19722 Meta query statistics metrics source 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-06-28 17:17:23 -07:00
zhangduo 0789e15b5e HBASE-20790 Fix the style issues on branch HBASE-19064 before merging back to master 2018-06-28 18:08:43 +08:00
zhangduo a84cdbd579 HBASE-20783 Addendum fix broken TestSyncReplicationStandBy 2018-06-28 18:08:43 +08:00
Guanghao Zhang 44ca13fe07 HBASE-20569 NPE in RecoverStandbyProcedure.execute 2018-06-28 18:08:43 +08:00
zhangduo 7448b045cc HBASE-20660 Reopen regions using ReopenTableRegionsProcedure 2018-06-28 18:08:43 +08:00
zhangduo 05295abd5b HBASE-20637 Polish the WAL switching when transiting from A to S 2018-06-28 18:08:43 +08:00
zhangduo f67763ffa0 HBASE-20424 Allow writing WAL to local and remote cluster concurrently 2018-06-28 18:08:43 +08:00
zhangduo 603110719d HBASE-20576 Check remote WAL directory when creating peer and transiting peer to A 2018-06-28 18:08:43 +08:00
zhangduo 8a264dfc00 HBASE-19865 Add UT for sync replication peer in DA state 2018-06-28 18:08:43 +08:00
zhangduo ae6c90b4ec HBASE-20426 Give up replicating anything in S state 2018-06-28 18:08:43 +08:00
huzheng 5b6c0d2777 HBASE-20432 Cleanup related resources when remove a sync replication peer 2018-06-28 18:08:43 +08:00
Guanghao Zhang 1bea678ef8 HBASE-20458 Support removing a WAL from LogRoller 2018-06-28 18:08:43 +08:00
zhangduo 2d203c4479 HBASE-20434 Also remove remote wals when peer is in DA state 2018-06-28 18:08:43 +08:00
zhangduo b281328228 HBASE-20456 Support removing a ReplicationSourceShipper for a special wal group 2018-06-28 18:08:43 +08:00
huzheng 66cced16dc HBASE-20425 Do not write the cluster id of the current active cluster when writing remote WAL 2018-06-28 18:08:43 +08:00
huzheng fe339860b5 HBASE-19782 Reject the replication request when peer is DA or A state 2018-06-28 18:08:43 +08:00
zhangduo d91784e666 HBASE-20370 Also remove the wal file in remote cluster when we finish replicating a file 2018-06-28 18:08:43 +08:00
Guanghao Zhang d57c80c415 HBASE-20163 Forbid major compaction when standby cluster replay the remote wals 2018-06-28 18:08:43 +08:00
zhangduo 2389c09d75 HBASE-19079 Support setting up two clusters with A and S stat 2018-06-28 18:08:43 +08:00
Guanghao Zhang c7d1085fa2 HBASE-19999 Remove the SYNC_REPLICATION_ENABLED flag 2018-06-28 18:07:44 +08:00
Guanghao Zhang 183b8d0581 HBASE-19973 Implement a procedure to replay sync replication wal for standby cluster 2018-06-28 18:07:44 +08:00
huzheng 45794d4156 HBASE-19943 Only allow removing sync replication peer which is in DA state 2018-06-28 18:07:44 +08:00
zhangduo 0c97cda2a9 HBASE-19990 Create remote wal directory when transitting to state S 2018-06-28 18:07:44 +08:00
zhangduo a41c549ca4 HBASE-19082 Reject read/write from client but accept write from replication in state S 2018-06-28 18:07:44 +08:00
zhangduo 39dd81a7c6 HBASE-19957 General framework to transit sync replication state 2018-06-28 18:07:44 +08:00
Guanghao Zhang 00e54aae24 HBASE-19935 Only allow table replication for sync replication for now 2018-06-28 18:07:44 +08:00
Guanghao Zhang 1481bd9481 HBASE-19864 Use protobuf instead of enum.ordinal to store SyncReplicationState 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-28 18:07:44 +08:00
zhangduo d8842dc3d4 HBASE-19857 Complete the procedure for adding a sync replication peer 2018-06-28 18:07:44 +08:00
Guanghao Zhang 2acebac00e HBASE-19781 Add a new cluster state flag for synchronous replication 2018-06-28 18:07:44 +08:00
zhangduo 274b813e12 HBASE-19747 Introduce a special WALProvider for synchronous replication 2018-06-28 18:07:44 +08:00
Guanghao Zhang b4a1dbf768 HBASE-19078 Add a remote peer cluster wal directory config for synchronous replication 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-28 18:07:44 +08:00
zhangduo b3dea0378e HBASE-19083 Introduce a new log writer which can write to two HDFSes 2018-06-28 18:07:44 +08:00
Michael Stack c23e61f20d HBASE-20781 Save recalculating families in a WALEdit batch of Cells 
Pass the Set of families through to the WAL rather than recalculate
a Set already known.

Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-27 22:04:57 -07:00
Reid Chan 74e5c776b3 HBASE-20732 Shutdown scan pool when master is stopped 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-28 11:42:18 +08:00
tedyu a8b16ac907 HBASE-20798 Duplicate thread names of StoreFileOpenerThread and StoreFileCloserThread (Zephyr Guo) 2018-06-27 17:21:07 -07:00
Sahil Aggarwal 952bb96c8a
HBASE-19164: Remove UUID.randomUUID in tests. 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-27 10:34:16 -05:00
jingyuntian 6a0c67344a HBASE-20194 Basic Replication WebUI - Master 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-26 18:26:54 +08:00
Michael Stack 4ba6242a62 HBASE-20780 ServerRpcConnection logging cleanup Get rid of one of the logging lines in ServerRpcConnection by amalgamating all into one new-style log line. 2018-06-25 16:43:11 -07:00
Michael Stack 0db2b628d6
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do 
General log cleanup; setting stuff that can flood the log to TRACE.
 2018-06-25 12:13:04 -07:00
Todd Lipcon 025ddce868 HBASE-20403. Fix race between prefetch task and non-pread HFile reads 
With prefetch-on-open enabled, the task doing the prefetching was using
non-positional (i.e. streaming) reads. If the main (non-prefetch) thread
was also using non-positional reads, these two would conflict, because
inputstreams are not thread-safe for non-positional reads.

In the case of an encrypted filesystem, this could cause JVM crashes,
etc, as underlying cipher buffers were freed underneath the racing
threads. In the case of a non-encrypted filesystem, less severe errors
would be thrown. The included unit test reproduces the latter case.
 2018-06-25 11:54:52 -07:00
zhangduo 9640ebacd4 HBASE-20777 RpcConnection could still remain opened after we shutdown the NettyRpcServer 2018-06-25 14:15:15 +08:00
Michael Stack daad14428d HBASE-20778 Make it so WALPE runs on DFS 2018-06-23 23:33:53 -07:00
zhangduo 55147c7eae HBASE-20775 Addendum disable REGIONS_ON_MASTER for TEstMultiParallel 2018-06-23 17:38:50 +08:00
zhangduo 14087cc919 HBASE-20775 TestMultiParallel is flakey 2018-06-22 21:32:07 +08:00
zhangduo 177458d9d0 HBASE-18569 Add prefetch support for async region locator 2018-06-22 18:25:31 +08:00
tedyu 98245ca6e4 HBASE-20740 StochasticLoadBalancer should consider CoprocessorService request factor when computing cost (chenxu) 2018-06-22 00:26:14 -07:00
zhangduo 7b716c964b HBASE-20752 Make sure the regions are truly reopened after ReopenTableRegionsProcedure 2018-06-22 14:04:33 +08:00
zhangduo 0d784efc37 HBASE-20767 Always close hbaseAdmin along with connection in HBTU 2018-06-21 21:01:19 +08:00
Ankit Singhal 72784c2d83 HBASE-20642 Clients should re-use the same nonce across DDL operations 
Also changes modify table operations to help the case where a MTP spans
two master, avoiding the sanity-checks propagating back to the client
unnecessarily.

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-06-20 14:56:10 -07:00
Josh Elser e989a9927e HBASE-20706 Prevent MTP from trying to reopen non-OPEN regions 
ModifyTableProcedure is using MoveRegionProcedure in a way
that was unintended from the original implementation. As such,
we have to guard against certain usages of it. We know we can
re-open OPEN regions, but regions in OPENING will similarly
soon be OPEN (thus, we want to reopen those regions too).

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-20 14:19:28 -07:00
zhangduo 4cb70ea9f5 HBASE-20739 Add priority for SCP 2018-06-20 15:17:07 +08:00
zhangduo c08eff67af HBASE-20742 Always create WAL directory for region server 2018-06-20 14:21:23 +08:00
Michael Stack 21684a32fa HBASE-20745 Log when master proc wal rolls 2018-06-19 19:53:51 -07:00
zhangduo 6dbbd78aa0 HBASE-20708 Remove the usage of RecoverMetaProcedure in master startup 2018-06-19 15:02:10 +08:00
Allan Yang b336da925a HBASE-20727 Persist FlushedSequenceId to speed up WAL split after cluster restart 2018-06-19 09:45:47 +08:00
Sean Busbey f1b536bad4 HBASE-20332 shaded mapreduce module shouldn't include hadoop 
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports
 2018-06-18 11:31:04 -07:00
tedyu ac5bb8155b HBASE-20723 Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them 2018-06-15 19:40:48 -07:00
taiynlee 0e43abc78a HBASE-20737 put collection into ArrayList instead of addAll function 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-16 03:25:42 +08:00
Xu Cang 86653c708f HBASE-20695 Implement table level RegionServer replication metrics 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-06-15 10:38:49 +08:00
jingyuntian 0b28155d27 HBASE-20625 refactor some WALCellCodec related code 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-06-14 19:37:01 +08:00
zhangduo 423a0ab71a HBASE-20722 Make RegionServerTracker only depend on children changed event 2018-06-14 08:36:37 +08:00
Guanghao Zhang ec66434380 HBASE-20561 The way we stop a ReplicationSource may cause the RS down 2018-06-13 17:58:59 +08:00
tedyu edf60b965b HBASE-20672 Adding new Metrics readRequestRate and writeRequestRate - revert pending discussion 2018-06-11 18:47:30 -07:00
Balazs Meszaros c323e7bfaa
HBASE-20656 Validate pre-2.0 coprocessors against HBase 2.0+ 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-11 10:26:58 -05:00
Mike Drob eb13cdd7ed HBASE-20707 Move MissingSwitchDefault case check 
Perform this check using error-prone instead of checkstyle because the
former can handle enum switches somewhat more intelligently.
 2018-06-11 09:57:50 -05:00
zhangduo 573b57d437 HBASE-20700 Move meta region when server crash can cause the procedure to be stuck 2018-06-11 14:57:31 +08:00
Guanghao Zhang cc7aefe0bb HBASE-20698 (addendum) Master don't record right server version until new started region server call regionServerReport method 2018-06-10 08:23:28 +08:00
Guanghao Zhang 5fd16f3853 HBASE-20698 Master don't record right server version until new started region server call regionServerReport method 2018-06-09 14:40:43 +08:00
Ankit 519236b4af HBASE-20672 Adding new Metrics readRequestRate and writeRequestRate 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-06-08 13:48:33 -07:00