Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,439 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

6947 Commits

Author SHA1 Message Date
Michael Stack 66add55234 HBASE-21072 Block out HBCK1 in hbase2 
Write the hbase-1.x hbck1 lock file to block out hbck1 instances writing
state to an hbase-2.x cluster (could do damage).
Set hbase.write.hbck1.lock.file to false to disable this writing.
 2018-08-24 09:22:53 -07:00
Duo Zhang 8a9acd4d2a HBASE-21101 Remove the waitUntilAllRegionsAssigned call after split in TestTruncateTableProcedure 2018-08-24 10:35:10 +08:00
zhangduo bf21a9dc33 HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file 2018-08-24 10:09:31 +08:00
Duo Zhang 239d12dae8 HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure 2018-08-23 06:27:41 +08:00
Allan Yang 16ab716134 HBASE-21041 Memstore's heap size will be decreased to minus zero after flush 2018-08-22 22:54:14 +08:00
Allan Yang c07afa8875 HBASE-21031 Memory leak if replay edits failed during region opening 2018-08-22 22:13:26 +08:00
Andrey Elenskiy 5f03be4675 HBASE-21032 ScanResponses contain only one cell each 
Amending-Author: Duo Zhang <zhangduo@apache.org>
 2018-08-21 13:31:18 -07:00
Andrew Purtell 798cb1d793 HBASE-20940 HStore.cansplit should not allow split to happen if it has references (Vishal Khandelwal) 2018-08-17 15:02:26 -07:00
Josh Elser 67ad0e6013 HBASE-21062 Correctly use the defaultProvider value on the Providers enum when constructing a WALProvider 2018-08-17 14:55:42 -04:00
Sakthi 48dee7e44d HBASE-20705 Having RPC quota on a table now no longer prevents Space Quota to be recreate/removed 
Just added 2 test cases as the subtasks of this jira solves the issue

Signed-off-by: Josh Elser <elserj@apache.org>
 2018-08-17 14:09:26 -04:00
Andrew Purtell b49941012a HBASE-21047 Object creation of StoreFileScanner thru constructor and close may leave refCount to -1 (Vishal Khandelwal) 2018-08-16 11:42:54 -07:00
Nihal Jain 145c92f3d6 HBASE-20469 Directory used for sidelining old recovered edits files should be made configurable 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-08-15 18:08:15 -07:00
Michael Stack 2e5efa690a HBASE-20772 Controlled shutdown fills Master log with the disturbing message 'No matching procedure found for rit=OPEN, location=ZZZZ, table=YYYYY, region=XXXX transition to CLOSED' 
Look for the particular case where RS does the close of region w/o
involving Master and log special message in this case. Dodgy. But
until we have Master run shutdown of all regions, better than
the message we currently show.
 2018-08-13 15:59:39 -07:00
Allan Yang 161c018927 HBASE-21029 Miscount of memstore's heap/offheap size if same cell was put 2018-08-13 20:30:23 +08:00
jingyuntian 95e3dec510 HBASE-20985 add two attributes when we do normalization 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-13 16:55:19 +08:00
Duo Zhang 846078f9b0 HBASE-21025 Addendum missed a 'succ = true' 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-13 10:55:18 +08:00
brandboat 873d9f5082 HBASE-21012 Revert the change of serializing TimeRangeTracker 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-08-11 22:28:49 +08:00
Wei-Chiu Chuang 5e12d6a98e HBASE-21018 RS crashed because AsyncFS was unable to update HDFS data encryption key 2018-08-10 19:53:22 -07:00
zhangduo ee164fcbc5 HBASE-21025 Add cache for TableStateManager 2018-08-10 21:11:53 +08:00
brandboat 8a9ba0c65b HBASE-18201 add UT and docs for DataBlockEncodingTool 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-08-10 11:19:36 +08:00
meiyi e222686294 HBASE-20965 Separate region server report requests to new handlers 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-09 18:27:38 +08:00
Sakthi b2fc0f48f6 HBASE-20813 Removed RPC quotas when the associated table/Namespace is dropped off 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-08-08 13:46:25 -04:00
jingyuntian 9d594ac86a HBASE-20986 Separate the config of block size when we do log splitting and write Hlog 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-07 14:03:03 +08:00
Sakthi 7e9f8c60e2 HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-08-03 11:07:01 -04:00
TAK LON WU 2e1c12ca1b HBASE-20856 PITA having to set WAL provider in two places 
With this change if hbase.wal.meta_provider is not explicitly set,
it uses whatever set with hbase.wal.provider. this change avoids a use
case of unexpectedly using two different providers when only
hbase.wal.provider is set to non-default but not hbase.wal.meta_provider.

This change also include document (architecture.adoc) update

Also, this is a port from master to branch-2

Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <Apache9@apache.org>
 2018-08-01 14:45:11 -07:00
Michael Stack 88f3148810 HBASE-20989 Minor, miscellaneous logging fixes 
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-08-01 11:20:01 -07:00
Xu Cang 9338eaee65 HBASE-20794 add INFO level log to createTable operation 2018-08-01 11:04:00 -07:00
Michael Stack 0f4e857c7a HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing ADDENDUM: Rather than rollback, just do region reopens. 
In split, reopen the parent if recovered.edits and in merge, reopen the
parent region or regions that happened to have recovered.edits on close.
 2018-08-01 00:33:12 -07:00
Andrew Purtell daeec8657e HBASE-20935 HStore.removeCompactedFiles should log in case it is unable to delete a file (Vishal Khandelwal) 
Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
 2018-07-31 16:06:15 -07:00
zhangduo 1b66839bc4 HBASE-20538 Upgrade our hadoop versions to 2.7.7 and 3.0.3 2018-07-29 20:20:28 +08:00
zhangduo a9346982bf Revert "HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3" 
This reverts commit 3dd83adb51.
 2018-07-29 20:20:20 +08:00
zhangduo 3dd83adb51 HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3 2018-07-29 20:04:48 +08:00
Alex Leblang 31cbd7ab8f
HBASE-19369 Switch to Builder Pattern In WAL 
This patch switches to the builder pattern by adding a helper method.
It also checks to ensure that the pattern is available (i.e. that
HBase is running on a hadoop version that supports it).

Amending-Author: Mike Drob <mdrob@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-27 23:43:08 -05:00
zhangduo 8bfdb19e85 HBASE-20939 There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException 2018-07-27 21:30:23 +08:00
Allan Yang 35c598db93 HBASE-20921 Possible NPE in ReopenTableRegionsProcedure 2018-07-27 09:31:12 +08:00
zhangduo 1777ea3aae HBASE-20938 Set version to 2.1.1-SNAPSHOT for branch-2.1 2018-07-25 21:45:09 +08:00
Allan Yang 3251554737 HBASE-20867 RS may get killed while master restarts 2018-07-25 18:11:28 +08:00
zhangduo 833657c46d HBASE-20846 Restore procedure locks when master restarts 2018-07-25 14:37:36 +08:00
huzheng 1dbfe92dbf HBASE-20565 ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result 2018-07-24 10:39:36 +08:00
Allan Yang 44bf7076b7 HBASE-20878 Data loss if merging regions while ServerCrashProcedure executing 2018-07-24 09:51:46 +08:00
Allan Yang af2742fcf2 HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing 2018-07-23 14:35:27 +08:00
Reid Chan 9d481f1faa HBASE-20401 Make MAX_WAIT and waitIfNotFinished in CleanerContext configurable (Contributed by Stephen Wu) 2018-07-23 10:33:26 +08:00
Michael Stack 46e5baf670 HBASE-20914 Trim Master memory usage 
Add (weak reference) interning of ServerNames.

Correct Balancer regions x racks matrix.

Make smaller defaults when creating ArrayDeques.
 2018-07-20 10:08:13 -07:00
Allan Yang 679698a7f2 HBASE-20870 Wrong HBase root dir in ITBLL's Search Tool 2018-07-20 11:31:21 +08:00
Yu Li 9ac26b80b2 HBASE-20907 Fix Intermittent failure on TestProcedurePriority 2018-07-19 12:01:29 +08:00
Michael Stack cecce16fad HBASE-20875 MemStoreLABImp::copyIntoCell uses 7% CPU when writing 
Make the #copyCellInto method smaller so it inlines; we do it by
checking for the common type early and then taking a code path
that presumes ByteBufferExtendedCell -- avoids checks.
 2018-07-18 20:40:47 -07:00
Toshihiro Suzuki e14b49080b HBASE-20865 CreateTableProcedure is stuck in retry loop in CREATE_TABLE_WRITE_FS_LAYOUT state 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-13 08:31:59 -07:00
Toshihiro Suzuki 881409bd0b HBASE-19572 RegionMover should use the configured default port number and not the one from HConstants 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-13 10:46:51 +08:00
Allan Yang 368b1b1060 HBASE-20860 Merged region's RIT state may not be cleaned after master restart 2018-07-12 12:16:49 +08:00
zhangduo 8eab6d7a45 HBASE-20847 Addendum use addFront instead of addBack to add sub procedure 2018-07-12 08:31:40 +08:00
zhangduo 113652eb88 HBASE-20847 The parent procedure of RegionTransitionProcedure may not have the table lock 2018-07-11 17:37:27 +08:00
zhaoyuan 8de69db143 HBASE-20697 Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations() 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-07-11 11:17:41 +08:00
zhangduo 5e25bc92cf HBASE-20784 Will lose the SNAPSHOT suffix if we get the version of RS from ServerManager 2018-07-10 10:00:15 +08:00
Abhishek Singh Chouhan dddf15ae6b HBASE-20806 Split style journal for flushes and compactions 2018-07-09 12:42:20 -07:00
Balazs Meszaros da7fef6bf5
HBASE-20833 Modify pre-upgrade coprocessor validator to support table level coprocessors 
- -jar parameter now accepts multiple jar files and directories of jar files.
- observer classes can be verified by -class option.
- -table parameter was added to check table level coprocessors.
- -config parameter was added to obtain the coprocessor classes from
  HBase cofiguration.
- -scan option was removed.

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-07-09 14:19:12 -05:00
zhangduo 5a40606422 HBASE-20822 TestAsyncNonMetaRegionLocator is flakey 2018-07-09 14:56:45 +08:00
Nihal Jain 927ac8228f HBASE-20808 (Addendum) Remove duplicate calls for cancelling of chores 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-07 00:21:08 +08:00
Nihal Jain 3ed9350233 HBASE-20808 Wrong shutdown order between Chores and ChoreService 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-07 00:20:08 +08:00
zhangduo a2db3d27ff HBASE-20849 Set version as 2.1.0 in branch-2.1 in prep for first RC 2018-07-06 15:32:23 +08:00
zhangduo 159f1b4686 Revert "HBASE-20808 Wrong shutdown order between Chores and ChoreService" 
For cutting 2.1.0RC0

This reverts commit ae2c858c5e.
 2018-07-06 15:29:58 +08:00
Nihal Jain ae2c858c5e HBASE-20808 Wrong shutdown order between Chores and ChoreService 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-06 11:38:17 +08:00
Yu Li d61bb64e93 HBASE-20691 Change the default WAL storage policy back to "NONE"" 
This reverts commit 564c193d61 and added more doc
about why we choose "NONE" as the default.
 2018-07-04 13:45:54 +08:00
Guangxu Cheng 60ebdd9fd8 HBASE-20474 Show non-RPC tasks on master/regionserver Web UI by default 2018-07-04 10:54:21 +08:00
zhangduo 5dacfe9427 HBASE-20839 Fallback to FSHLog if we can not instantiated AsyncFSWAL when user does not specify AsyncFSWAL explicitly 2018-07-04 10:29:36 +08:00
zhangduo fedbd00ef1 HBASE-20829 Remove the addFront assertion in MasterProcedureScheduler.doAdd 2018-07-04 09:41:02 +08:00
Ted Yu 927a957390 HBASE-20244 NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 22:16:04 +08:00
huzheng ce99588530 HBASE-20789 TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky 2018-07-03 18:05:17 +08:00
jingyuntian 6b8cd00ec0 HBASE-20193 Basic Replication Web UI - Regionserver 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 16:10:31 +08:00
Josh Elser 8f9c322cda HBASE-20826 Truncate really long RpcServer warnings unless TRACE is on 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-07-03 10:14:57 +08:00
Ankit Singhal d22c6de648 HBASE-20825 Fix pre and post hooks of CloneSnapshot and RestoreSnapshot for Access checks 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 10:06:13 +08:00
Ankit Singhal 4a80b19d7f HBASE-20817 Infinite loop when executing ReopenTableRegionsProcedure 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-02 21:29:09 +08:00
Josh Elser 44573b54c1 HBASE-20792 info:servername and info:sn inconsistent for OPEN region 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-06-29 11:11:22 +08:00
Michael Stack becb638370 HBASE-20781 Save recalculating families in a WALEdit batch of Cells 
Pass the Set of families through to the WAL rather than recalculate
a Set already known.

Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-27 22:04:22 -07:00
Reid Chan 43c0df51ea HBASE-20732 Shutdown scan pool when master is stopped 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-28 12:54:54 +08:00
Sahil Aggarwal 4ba2abf43b
HBASE-19164: Remove UUID.randomUUID in tests. 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-27 10:37:15 -05:00
jingyuntian bd40cba8dd HBASE-20194 Basic Replication WebUI - Master 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-26 18:29:01 +08:00
zhangduo 07afb7e32f HBASE-20777 RpcConnection could still remain opened after we shutdown the NettyRpcServer 2018-06-26 09:08:05 +08:00
Michael Stack 3f319bef8d HBASE-20780 ServerRpcConnection logging cleanup Get rid of one of the logging lines in ServerRpcConnection by amalgamating all into one new-style log line. 2018-06-25 16:44:07 -07:00
Todd Lipcon 3673bfc241 HBASE-20403. Fix race between prefetch task and non-pread HFile reads 
With prefetch-on-open enabled, the task doing the prefetching was using
non-positional (i.e. streaming) reads. If the main (non-prefetch) thread
was also using non-positional reads, these two would conflict, because
inputstreams are not thread-safe for non-positional reads.

In the case of an encrypted filesystem, this could cause JVM crashes,
etc, as underlying cipher buffers were freed underneath the racing
threads. In the case of a non-encrypted filesystem, less severe errors
would be thrown. The included unit test reproduces the latter case.

(cherry picked from commit 025ddce868)
Signed-off-by: Todd Lipcon <todd@cloudera.com>
 2018-06-25 12:12:30 -07:00
Michael Stack d6cea08efe
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do 
General log cleanup; setting stuff that can flood the log to TRACE.
 2018-06-25 12:12:03 -07:00
Michael Stack 7c45f02110 HBASE-20778 Make it so WALPE runs on DFS 2018-06-23 23:34:40 -07:00
zhangduo eb67404cef HBASE-20775 TestMultiParallel is flakey 2018-06-24 08:42:53 +08:00
zhangduo f3061a67fc HBASE-18569 Add prefetch support for async region locator 2018-06-22 18:20:21 +08:00
zhangduo a86141b625 HBASE-20752 Make sure the regions are truly reopened after ReopenTableRegionsProcedure 2018-06-22 14:06:29 +08:00
zhangduo 6cebe06225 HBASE-20767 Always close hbaseAdmin along with connection in HBTU 2018-06-22 10:20:06 +08:00
Ankit Singhal 28d0d8c5cc HBASE-20642 Clients should re-use the same nonce across DDL operations 
Also changes modify table operations to help the case where a MTP spans
two master, avoiding the sanity-checks propagating back to the client
unnecessarily.

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-06-20 15:10:52 -07:00
Josh Elser c8b76eb3f1 HBASE-20706 Prevent MTP from trying to reopen non-OPEN regions 
ModifyTableProcedure is using MoveRegionProcedure in a way
that was unintended from the original implementation. As such,
we have to guard against certain usages of it. We know we can
re-open OPEN regions, but regions in OPENING will similarly
soon be OPEN (thus, we want to reopen those regions too).

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-20 14:30:36 -07:00
zhangduo 5c2cb15e0b HBASE-20739 Add priority for SCP 2018-06-20 15:18:47 +08:00
zhangduo 83969b0da1 HBASE-20742 Always create WAL directory for region server 2018-06-20 14:21:28 +08:00
Michael Stack 9eeb501825 HBASE-20745 Log when master proc wal rolls 2018-06-19 19:53:29 -07:00
zhangduo 3e33aecea2 HBASE-20708 Remove the usage of RecoverMetaProcedure in master startup 2018-06-19 15:09:11 +08:00
Sean Busbey ee84a8f243 HBASE-20332 shaded mapreduce module shouldn't include hadoop 
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports

 Conflicts:
	hbase-backup/pom.xml
	hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-18 14:02:48 -07:00
Mike Drob b04c976fe6 HBASE-20478 Update checkstyle to v8.2 
Cannot go to latest (8.9) yet due to
  https://github.com/checkstyle/checkstyle/issues/5279

* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors

Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-06-18 14:02:40 -07:00
taiynlee 8edd5d948a HBASE-20737 put collection into ArrayList instead of addAll function 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-17 11:16:16 +08:00
tedyu b2afba580b HBASE-20723 Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them 2018-06-16 01:34:53 -07:00
Xu Cang b68746c0b2 HBASE-20695 Implement table level RegionServer replication metrics 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-06-15 10:45:13 +08:00
jingyuntian bde9f08a83 HBASE-20625 refactor some WALCellCodec related code 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-06-14 19:46:33 +08:00
zhangduo 161dc7c7f3 HBASE-20722 Make RegionServerTracker only depend on children changed event 2018-06-14 08:38:53 +08:00
Guanghao Zhang 075523dd1e HBASE-20561 The way we stop a ReplicationSource may cause the RS down 2018-06-13 18:05:27 +08:00
Balazs Meszaros d44e8a7aff HBASE-20656 Validate pre-2.0 coprocessors against HBase 2.0+ 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-11 10:32:40 -05:00