Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,459 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

6967 Commits

Author SHA1 Message Date
Michael Stack a83073aff0 HBASE-21078 [amv2] CODE-BUG NPE in RTP doing Unassign 2018-08-24 13:22:45 -07:00
Michael Stack 6047fb05f5 HBASE-21113 Apply the branch-2 version of HBASE-21095, The timeout retry logic for several procedures are broken after master restarts 
I applied the patch HBASE-21095 and then reverted it so could apply the
patch as HBASE-21113 (by reverting the HBASE-21095 revert but pushing
with this message!).

Revert "Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts""

This reverts commit a220566b98.
 2018-08-24 12:32:33 -07:00
Michael Stack a220566b98 Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts" 
This reverts commit 09be1d0d57.
 2018-08-24 12:24:17 -07:00
Allan Yang 09be1d0d57 HBASE-21095 The timeout retry logic for several procedures are broken after master restarts 2018-08-24 12:21:01 -07:00
Michael Stack 2ea45e67d0 HBASE-21072 Block out HBCK1 in hbase2 
Write the hbase-1.x hbck1 lock file to block out hbck1 instances writing
state to an hbase-2.x cluster (could do damage).
Set hbase.write.hbck1.lock.file to false to disable this writing.
 2018-08-24 09:23:54 -07:00
Duo Zhang c33af1e854 HBASE-21101 Remove the waitUntilAllRegionsAssigned call after split in TestTruncateTableProcedure 2018-08-24 10:35:05 +08:00
zhangduo b318311dfd HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file 2018-08-24 10:09:23 +08:00
tedyu 87f9b4accd HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning 2018-08-23 11:49:47 -07:00
liubangchen a7a281a644 HBASE-19008 Add missing equals or hashCode method(s) to stock Filter implementations 
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
 2018-08-24 00:39:35 +08:00
Duo Zhang cc6fc1f04f HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure 2018-08-23 06:27:37 +08:00
Allan Yang a7da458c5b HBASE-21041 Memstore's heap size will be decreased to minus zero after flush 2018-08-22 23:03:20 +08:00
Allan Yang 77e1a66f1e HBASE-21031 Memory leak if replay edits failed during region opening 2018-08-22 22:22:35 +08:00
Xu Cang b5ae4e5f9e HBASE-20666 Unsuccessful table creation leaves entry in hbase:rsgroup table 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-08-21 18:55:46 -07:00
Andrey Elenskiy d1b5113511 HBASE-21032 ScanResponses contain only one cell each 
Amending-Author: Duo Zhang <zhangduo@apache.org>
 2018-08-21 13:31:49 -07:00
tedyu 3787c72953 HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region - addendum 2018-08-19 14:25:12 -07:00
Andrew Purtell 52ba33aa73 HBASE-20940 HStore.cansplit should not allow split to happen if it has references (Vishal Khandelwal) 2018-08-17 15:02:22 -07:00
Andrew Purtell 1fa67725c5 HBASE-21047 Object creation of StoreFileScanner thru constructor and close may leave refCount to -1 (Vishal Khandelwal) 2018-08-16 11:42:49 -07:00
Sakthi 75939775af HBASE-20705 Having RPC quota on a table now no longer prevents Space Quota to be recreate/removed 
Just added 2 test cases as the subtasks of this jira solves the issue

Signed-off-by: Josh Elser <elserj@apache.org>
 2018-08-16 11:16:56 -04:00
Josh Elser cbe2fc113d HBASE-21062 Correctly use the defaultProvider value on the Providers enum when constructing a WALProvider 2018-08-16 10:18:39 -04:00
Nihal Jain d5abb92088 HBASE-20469 Directory used for sidelining old recovered edits files should be made configurable 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-08-15 18:01:24 -07:00
jingyuntian bee8566f2c HBASE-20985 add two attributes when we do normalization 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-13 16:49:37 +08:00
Duo Zhang d979f70142 HBASE-21025 Addendum missed a 'succ = true' 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-13 10:54:49 +08:00
brandboat 51df2887c4 HBASE-21012 Revert the change of serializing TimeRangeTracker 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-08-11 22:25:10 +08:00
zhangduo 2311fa86d4 HBASE-21025 Add cache for TableStateManager 2018-08-10 21:11:45 +08:00
brandboat fb9aa8a95a HBASE-18201 add UT and docs for DataBlockEncodingTool 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-08-10 11:16:36 +08:00
meiyi 28c3336bfe HBASE-20965 Separate region server report requests to new handlers 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-09 18:15:22 +08:00
Sakthi 6a8bd0fe84 HBASE-20813 Removed RPC quotas when the associated table/Namespace is dropped off 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-08-07 13:43:27 -04:00
jingyuntian 1aacde4839 HBASE-20986 Separate the config of block size when we do log splitting and write Hlog 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-08-07 14:00:31 +08:00
Sakthi b2cefab0e1 HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-08-03 11:06:56 -04:00
Monani Mihir 06a92a3d20 HBASE-19036 Add action in Chaos Monkey to restart Active Namenode 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-08-02 05:00:16 -07:00
TAK LON WU 690d29bae7 HBASE-20856 PITA having to set WAL provider in two places 
With this change if hbase.wal.meta_provider is not explicitly set,
it uses whatever set with hbase.wal.provider. this change avoids a use
case of unexpectedly using two different providers when only
hbase.wal.provider is set to non-default but not hbase.wal.meta_provider.

This change also include document (architecture.adoc) update

Also, this is a port from master to branch-2

Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <Apache9@apache.org>
 2018-08-01 14:36:29 -07:00
Michael Stack fc0c4660fa HBASE-20989 Minor, miscellaneous logging fixes 
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2018-08-01 11:20:26 -07:00
Xu Cang bfb9c71e3b HBASE-20794 add INFO level log to createTable operation 2018-08-01 11:04:18 -07:00
Michael Stack 477d567fd7 HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing ADDENDUM: Rather than rollback, just do region reopens. 
In split, reopen the parent if recovered.edits and in merge, reopen the
parent region or regions that happened to have recovered.edits on close.
 2018-08-01 00:33:33 -07:00
Andrew Purtell 777f3e88e2 HBASE-20935 HStore.removeCompactedFiles should log in case it is unable to delete a file (Vishal Khandelwal) 
Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
 2018-07-31 15:59:53 -07:00
Wei-Chiu Chuang 99714da30f HBASE-20950 Helper method to configure secure DFS cluster for tests 
Create a helper method HBaseKerberosUtils#setSecuredConfiguration().
TestSecureExport, TestSaslFanOutOneBlockAsyncDFSOutput,
SecureTestCluster and TestThriftSpnegoHttpServer uses this new helper
method.

Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-31 13:37:10 -07:00
Reid Chan e14b60a539 HBASE-20886 [Auth] Support keytab login in hbase client 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-07-31 11:10:34 +08:00
zhangduo 584093c23f HBASE-20538 Upgrade our hadoop versions to 2.7.7 and 3.0.3 2018-07-29 20:20:01 +08:00
zhangduo 5957fceda4 Revert "HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3" 
This reverts commit 71a4c17d99.
 2018-07-29 20:19:16 +08:00
zhangduo 71a4c17d99 HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3 2018-07-29 20:04:53 +08:00
Alex Leblang ab18440107
HBASE-19369 Switch to Builder Pattern In WAL 
This patch switches to the builder pattern by adding a helper method.
It also checks to ensure that the pattern is available (i.e. that
HBase is running on a hadoop version that supports it).

Amending-Author: Mike Drob <mdrob@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-27 23:42:53 -05:00
zhangduo 51b0d0edd0 HBASE-20939 There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException 2018-07-27 21:25:22 +08:00
Allan Yang 611ca11c9f HBASE-20921 Possible NPE in ReopenTableRegionsProcedure 2018-07-27 09:34:25 +08:00
Mingliang Liu bb6f24cd59 HBASE-20932 Effective MemStoreSize::hashCode() 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-25 14:42:47 -07:00
Peter Somogyi 3b84c9c8d2 HBASE-20649 Validate HFiles do not have PREFIX_TREE DataBlockEncoding 
Amending-Author: Balazs Meszaros <balazs.meszaros@cloudera.com>

Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-07-25 09:12:17 -05:00
Allan Yang ac4b789f5f HBASE-20867 RS may get killed while master restarts 2018-07-25 18:13:50 +08:00
zhangduo 0b283dae0d HBASE-20846 Restore procedure locks when master restarts 2018-07-25 14:37:31 +08:00
Saurabh Singh 097ae55708 HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-24 08:17:16 -07:00
huzheng 99701968c8 HBASE-20565 ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result 2018-07-24 10:33:33 +08:00
Allan Yang aaf5c78445 HBASE-20878 Data loss if merging regions while ServerCrashProcedure executing 2018-07-24 09:57:47 +08:00
Allan Yang 786fd9137a HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing 2018-07-23 14:46:32 +08:00
tedyu 654480326b HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region 2018-07-22 20:51:58 -07:00
Reid Chan dd0325a740 HBASE-20401 Make MAX_WAIT and waitIfNotFinished in CleanerContext configurable (Contributed by Stephen Wu) 2018-07-23 10:32:17 +08:00
Toshihiro Suzuki b15efbf8ed HBASE-19893 restore_snapshot is broken in master branch when region splits 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-21 07:47:50 -07:00
Ankit Singhal e8a7a4472f HBASE-20908 Infinite loop on regionserver if region replica are reduced 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-20 13:05:43 -07:00
Michael Stack 810473b277 HBASE-20914 Trim Master memory usage 
Add (weak reference) interning of ServerNames.

Correct Balancer regions x racks matrix.

Make smaller defaults when creating ArrayDeques.
 2018-07-20 10:08:37 -07:00
Allan Yang 1a6fae74b5 HBASE-20870 Wrong HBase root dir in ITBLL's Search Tool 2018-07-20 11:22:03 +08:00
Ankit 37e5f5fb5f HBASE-20672 New metrics ReadRequestRate and WriteRequestRate 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-07-19 15:41:53 -07:00
Ankit Singhal cf1f63bf91 HBASE-20901 Reducing region replica has no effect 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-19 13:39:44 -07:00
Mohit Goel 35a3c605f2 HBASE-6028 Start/Stop compactions at region server level 
Add switching on/off of compactions.

Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.

Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-07-19 06:20:12 -07:00
Yu Li e464230bde HBASE-20907 Fix Intermittent failure on TestProcedurePriority 2018-07-19 12:01:16 +08:00
Michael Stack 532d84d6d0 HBASE-20875 MemStoreLABImp::copyIntoCell uses 7% CPU when writing 
Make the #copyCellInto method smaller so it inlines; we do it by
checking for the common type early and then taking a code path
that presumes ByteBufferExtendedCell -- avoids checks.
 2018-07-18 20:41:05 -07:00
Allan Yang 3078acb83c HBASE-20860 Merged region's RIT state may not be cleaned after master restart 2018-07-17 17:33:11 +08:00
tedyu 07fcdf9418 HBASE-20879 Compacting memstore config should handle lower case 2018-07-13 11:22:06 -07:00
Toshihiro Suzuki d3bb364595 HBASE-20865 CreateTableProcedure is stuck in retry loop in CREATE_TABLE_WRITE_FS_LAYOUT state 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-13 08:26:05 -07:00
Toshihiro Suzuki d7f4b06206 HBASE-19572 RegionMover should use the configured default port number and not the one from HConstants 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-13 10:46:01 +08:00
zhangduo 59830b0078 HBASE-20847 Addendum use addFront instead of addBack to add sub procedure 2018-07-12 08:31:21 +08:00
zhangduo 430b056ade HBASE-20847 The parent procedure of RegionTransitionProcedure may not have the table lock 2018-07-11 17:37:21 +08:00
eshcar 176afd998a HBASE-20542-ADDENDUM: fix TestHStore 2018-07-11 11:31:28 +03:00
zhaoyuan 7990283d50 HBASE-20697 Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations() 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2018-07-11 11:07:04 +08:00
zhangduo 8decae9b92 HBASE-20784 Will lose the SNAPSHOT suffix if we get the version of RS from ServerManager 2018-07-10 10:00:11 +08:00
Balazs Meszaros e6403c18ff
HBASE-20833 Modify pre-upgrade coprocessor validator to support table level coprocessors 
- -jar parameter now accepts multiple jar files and directories of jar files.
- observer classes can be verified by -class option.
- -table parameter was added to check table level coprocessors.
- -config parameter was added to obtain the coprocessor classes from
  HBase cofiguration.
- -scan option was removed.

Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-07-09 11:15:18 -05:00
Abhishek Singh Chouhan ae22d57201 HBASE-20806 Split style journal for flushes and compactions 2018-07-09 18:42:30 +05:30
zhangduo 8a394ade4c HBASE-20822 TestAsyncNonMetaRegionLocator is flakey 2018-07-09 14:56:42 +08:00
Nihal Jain 1da2fa3baa HBASE-20808 (Addendum) Remove duplicate calls for cancelling of chores 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-07 00:18:22 +08:00
Nihal Jain 473c71edb1 HBASE-20808 Wrong shutdown order between Chores and ChoreService 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2018-07-06 11:37:13 +08:00
Yu Li 4653d4ac6b HBASE-20691 Change the default WAL storage policy back to "NONE"" 
This reverts commit 564c193d61 and added more doc
about why we choose "NONE" as the default.
 2018-07-04 13:45:16 +08:00
Guangxu Cheng 341fa55367 HBASE-20474 Show non-RPC tasks on master/regionserver Web UI by default 2018-07-04 10:56:46 +08:00
zhangduo 5ef9d8f451 HBASE-20839 Fallback to FSHLog if we can not instantiated AsyncFSWAL when user does not specify AsyncFSWAL explicitly 2018-07-04 10:29:30 +08:00
zhangduo c612642b2b HBASE-20829 Remove the addFront assertion in MasterProcedureScheduler.doAdd 2018-07-04 09:40:52 +08:00
Ted Yu 021eae48ca HBASE-20244 NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 22:15:48 +08:00
huzheng 8b1e6a81ff HBASE-20789 TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky 2018-07-03 17:59:25 +08:00
jingyuntian 30664b1a57 HBASE-20193 Basic Replication Web UI - Regionserver 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-03 16:02:28 +08:00
Josh Elser 33d002f445 HBASE-20826 Truncate really long RpcServer warnings unless TRACE is on 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-07-03 10:14:52 +08:00
Nihal Jain ad7e453817 HBASE-20450 Provide metrics for number of total active, priority and replication rpc handlers 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-07-02 11:38:02 -07:00
Ankit Singhal 3847def1c2 HBASE-20817 Infinite loop when executing ReopenTableRegionsProcedure 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-07-02 21:28:53 +08:00
eshcar 7868f0ef97 HBASE-20542: Better heap utilization for IMC with MSLABs 2018-07-01 15:52:30 +03:00
Ankit Singhal 490ca0ce20 HBASE-20825 Fix pre and post hooks of CloneSnapshot and RestoreSnapshot for Access checks 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
 2018-06-29 16:33:34 -04:00
Pankaj a16e5523fc HBASE-20357 AccessControlClient API Enhancement 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-06-29 07:54:14 -07:00
Josh Elser 02235f4e32 HBASE-20792 info:servername and info:sn inconsistent for OPEN region 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-06-29 11:11:09 +08:00
Xu Cang 58ccd3dc7e HBASE-19722 Meta query statistics metrics source 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-06-28 17:17:20 -07:00
Michael Stack f264b79ec1 HBASE-20781 Save recalculating families in a WALEdit batch of Cells 
Pass the Set of families through to the WAL rather than recalculate
a Set already known.

Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-27 22:04:40 -07:00
Reid Chan 32e0767dfd HBASE-20732 Shutdown scan pool when master is stopped 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-06-28 11:51:52 +08:00
Sahil Aggarwal e61507b9a0
HBASE-19164: Remove UUID.randomUUID in tests. 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-06-27 10:36:48 -05:00
jingyuntian 557985ee47 HBASE-20194 Basic Replication WebUI - Master 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-06-26 18:27:50 +08:00
zhangduo a1b297e673 HBASE-20777 RpcConnection could still remain opened after we shutdown the NettyRpcServer 2018-06-26 09:07:58 +08:00
Michael Stack 40cd76af8a HBASE-20780 ServerRpcConnection logging cleanup Get rid of one of the logging lines in ServerRpcConnection by amalgamating all into one new-style log line. 2018-06-25 16:43:43 -07:00
Michael Stack 4e16ecd7f4
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do 
General log cleanup; setting stuff that can flood the log to TRACE.
 2018-06-25 12:12:43 -07:00
Todd Lipcon bf65e95466 HBASE-20403. Fix race between prefetch task and non-pread HFile reads 
With prefetch-on-open enabled, the task doing the prefetching was using
non-positional (i.e. streaming) reads. If the main (non-prefetch) thread
was also using non-positional reads, these two would conflict, because
inputstreams are not thread-safe for non-positional reads.

In the case of an encrypted filesystem, this could cause JVM crashes,
etc, as underlying cipher buffers were freed underneath the racing
threads. In the case of a non-encrypted filesystem, less severe errors
would be thrown. The included unit test reproduces the latter case.

(cherry picked from commit 025ddce868)
Signed-off-by: Todd Lipcon <todd@cloudera.com>
 2018-06-25 12:11:36 -07:00
Michael Stack e915024e3c HBASE-20778 Make it so WALPE runs on DFS 2018-06-23 23:34:17 -07:00