87d4e4e4e2
HBASE-20646 TestWALProcedureStoreOnHDFS failing on branch-1
2018-05-24 18:06:12 -07:00
b892be7440
HBASE-20548 Master fails to startup on large clusters, refreshing block distribution
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-24 12:27:19 -07:00
1b70763b9e
HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-23 16:45:55 -07:00
6d1aa2800b
HBASE-20486 Change default throughput controller to PressureAwareThroughputController
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-22 12:43:31 -07:00
2d493556f3
HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics
...
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
"NaN", "Infinity" or "-Infinity"
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-16 11:50:31 -07:00
2a309d71cc
HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-15 18:17:52 -07:00
944a221b74
HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
...
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 16:09:36 -07:00
ca544a155c
HBASE-20004 Client is not able to execute REST queries in a secure cluster
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-05-10 22:49:08 +05:30
c191462ac3
HBASE-20554 "WALs outstanding" message from CleanerChore is noisy
2018-05-09 19:13:17 -07:00
5406f63332
HBASE-20204 Add locking to RefreshFileConnections in BucketCache
...
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
2018-05-09 14:15:02 -07:00
9d9d5aec0e
HBASE-20505 PE should support multi column family read and write cases
2018-05-07 18:39:18 -07:00
91f3de89ab
HBASE-20523 PE tool should support configuring client side buffering sizes
...
(Ram)
2018-05-07 14:41:08 +05:30
23b9054089
HBASE-20513 Collect and emit ScanMetrics in PerformanceEvaluation
2018-05-04 17:59:28 -07:00
fd24083e26
HBASE-20517 Fix PerformanceEvaluation 'column' parameter
2018-05-04 17:24:38 -07:00
e793e7c30c
HBASE-20493 Port HBASE-19994 (Create a new class for RPC throttling exception, make it retryable) to branch-1
2018-04-26 15:38:40 -07:00
71c573239b
HBASE-20496 TestGlobalThrottler failing on branch-1 since revert of HBASE-9465
...
Temporarily disable TestGlobalThrottler on branch-1
2018-04-26 15:38:40 -07:00
eb67d4e308
HBASE-19924 hbase rpc throttling does not work for multi() with request count rater.
2018-04-26 10:43:59 -07:00
f11cf007a2
HBASE-20229 ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on
2018-04-25 13:36:12 -07:00
e60189035e
HBASE-20406 HBase Thrift HTTP - Shouldn't handle TRACE/OPTIONS methods
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestHttpServer.java
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
2018-04-20 22:45:49 -05:00
eacf3cb296
Revert "HBase Thrift HTTP - Shouldn't handle TRACE/OPTIONS methods"
...
This reverts commit fe84833ea2
2018-04-20 22:45:37 -05:00
fe84833ea2
HBase Thrift HTTP - Shouldn't handle TRACE/OPTIONS methods
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestHttpServer.java
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
2018-04-20 15:15:31 -05:00
8a244e51d7
HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClientWithRegionReplicas.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
2018-04-20 11:34:45 -05:00
03eb3d24b6
HBASE-19850 The number of Offline Regions is wrong after restoring a snapshot
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-17 20:29:50 -07:00
ce4c243399
HBASE-20404 Fixes to CleanChore correctness and operability.
...
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestCleanerChore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java
2018-04-17 13:05:41 -05:00
a817f196a1
HBASE-15291 FileSystem not closed in secure bulkLoad
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-11 12:59:52 +05:30
427c95e2e5
HBASE-20352 [Chore] Backport HBASE-18309 (Support multi threads in CleanerChore) to branch-1
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-11 14:17:51 +08:00
eacbe002e5
HBASE-19343 Restore snapshot makes split parent region online
2018-04-09 07:48:34 -07:00
2f683cd438
HBASE-20301 Remove the meaningless plus sign from table.jsp
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-04 20:10:46 +08:00
9ced0c936f
Amend HBASE-20322 CME in StoreScanner causes region server crash
...
memStoreScanners is immutable so create a new list for closing.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-04-03 13:22:43 -07:00
875a12de05
HBASE-20322 CME in StoreScanner causes region server crash
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-04-02 19:36:16 -07:00
18c54b02b1
HBASE-17631 Changed Canary interval to 60 seconds
2018-04-01 13:48:28 +02:00
9a6b6524a7
HBASE-20302 CatalogJanitor should log the reason why it is disabled (Vishal Khandelwal)
2018-03-29 10:20:34 -07:00
773af3e0ca
HBASE-20280 Fix possibility of deadlocking in refreshFileConnections
...
When prefetch on open is specified, there is a deadlocking case
where if the prefetch is cancelled, the PrefetchExecutor interrupts
the threads if necessary, when that happens in FileIOEngine, it
causes an ClosedByInterruptException which is a subclass of
ClosedChannelException. If we retry all ClosedChannelExceptions,
this will lock as this access is expected to be interrupted.
This change removes calling refreshFileConnections for
ClosedByInterruptExceptions.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-03-27 22:59:58 +00:00
caf4c4b4ee
HBASE-20261 Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-23 13:34:45 -04:00
f976b3a8af
HBASE-19024 Provide a configurable option to hsync WAL edits to the disk for better durability (Harshal Jain)
2018-03-20 17:51:59 -07:00
009295a3a6
[HBASE-20141] Fix TooManyFiles exception when RefreshingChannels
...
HBASE-19435 implements a fix for reopening file channels when they are unnexpected closed
to avoid disabling the BucketCache. However, it was missed that the the channels might not
actually be completely closed (the write or read channel might still be open
(see https://docs.oracle.com/javase/7/docs/api/java/nio/channels/ClosedChannelException.html )
This commit closes any open channels before creating a new channel.
2018-03-16 10:58:22 -07:00
a0b2141a95
HBASE-20146 Addendum Regions are stuck while opening when WAL is disabled
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-15 10:17:03 +08:00
5aeddb3051
HBASE-18864 (addendum) Fixed unit test failure TestFromClientSide#testIllegalTableDescriptor
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-03-14 13:57:08 -07:00
38471229f9
HBASE-20174 Fix TestZKLessMergeOnCluster flakiness
2018-03-12 16:57:59 -07:00
e81abb7259
HBASE-20146 Regions are stuck while opening when WAL is disabled
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 21:27:00 +08:00
41ff229a16
HBASE-20139 NPE in RSRpcServices.get() when getRegion throws an exception
2018-03-07 12:09:04 +05:30
c4401b6073
HBASE-19400 Add missing security checks in MasterRpcServices
...
- Added ADMIN permission check for following rpc calls:
normalize, setNormalizerRunning, runCatalogScan, enableCatalogJanitor, runCleanerChore,
setCleanerChoreRunning, execMasterService, execProcedure, execProcedureWithRet
- Moved authorizationEnabled check to start of AccessChecker's functions. Currently, and IDK why,
we call authManager.authorize() first and then discard its result if authorizationEnabled is false. Weird.
----
HBASE-19401 Add missing security checks in RSRpcServices
2018-02-28 11:52:16 -08:00
cdbf001d26
HBASE-19863 java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: ramkrish86 <ramkrishna.s.vasudevan@gmail.com>
2018-02-28 13:35:43 -05:00
1be9a3d641
HBASE-20087 Periodically attempt redeploy of regions in FAILED_OPEN state
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-02-27 17:11:44 -08:00
9a7a36750f
HBASE-20102 AssignmentManager#shutdown doesn't shut down scheduled executor
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-02-27 17:11:28 -08:00
27f803f320
HBASE-19989 Addendum - temp override for tests to be added later for HBASE-20001
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-27 17:06:32 -08:00
40e0ee331c
HBASE-19989 - READY_TO_MERGE and READY_TO_SPLIT do not update region state correctly
...
Author: Ben Lau <benlau@oath.com>
2018-02-27 12:41:48 -08:00
ece68d3c65
HBASE-20001 cleanIfNoMetaEntry() uses encoded instead of region name to lookup region
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-26 18:01:28 -08:00
a37c91bb41
HBASE-20061 HStore synchronized member variable filesCompacting should be private
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
2018-02-24 10:59:18 -06:00
343cdda9ae
HBASE-19364, Truncate_preserve fails with table when replica region > 1
2018-02-23 14:13:02 -08:00
ba7a936f74
Revert "HBASE-9465 Push entries to peer clusters serially"
...
This reverts commit 441bc050b9
2018-02-23 14:42:15 +08:00
0a284d2b76
Revert "HBASE-17010 Serial replication should handle daughter regions being assigned to another RS (Phil Yang)"
...
This reverts commit 97276da9a7
2018-02-23 14:42:04 +08:00
4d9589b165
HBASE-20052 TestRegionOpen#testNonExistentRegionReplica fails due to NPE
2018-02-22 12:41:13 -08:00
cadc2efd12
HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory; ADDEDNDUM to fix TestRegionOpen failure
2018-02-22 09:27:46 -08:00
af1f7cf431
HBASE-20016 TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
2018-02-22 09:44:37 +08:00
a189674b4c
HBASE-19553, Old replica regions should be cleared from AM memory after primary region split or merge
...
Signed-off-by: Ted Yu <tedyu@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2018-02-21 16:52:40 -08:00
9e7e58d551
HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory
2018-02-21 16:25:13 -08:00
c5e692c1dc
HBASE-20027 Add test TestClusterPortAssignment
...
LocalHBaseCluster forces random port assignment for sake of concurrent unit test
execution friendliness, but we still need a positive test for RPC and info port
assignment.
2018-02-21 13:12:26 -08:00
c0bca137ff
HBASE-20008 [backport] NullPointerException when restoring a snapshot after splitting a region
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-20 20:29:08 -08:00
21f6830fe1
HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-20 16:59:44 -08:00
fd8189d31d
Revert "HBASE-19970 Remove unused functions from TableAuthManager."
...
This reverts commit e6ce789b6f
2018-02-20 11:12:36 -08:00
071281cf52
Revert "HBASE-19970 (addendum for 1.x only) Remove unused functions from TableAuthManager."
...
This reverts commit 0f79c497c5
2018-02-20 11:12:33 -08:00
7990546efe
HBASE-14897 TestTableLockManager.testReapAllTableLocks is flakey
...
(cherry picked from commit 47082579a8
2018-02-20 08:50:22 -06:00
28ebd29f0f
HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of a KeeperException
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-14 17:23:23 -08:00
e65004aeea
HBASE-19996 Some nonce procs might not be cleaned up (follow up HBASE-19756)
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-14 09:17:13 -08:00
335b8a8e14
HBASE-19979 ReplicationSyncUp tool may leak Zookeeper connection (Pankaj Kumar)
2018-02-13 08:11:03 -08:00
f77b42b2af
HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
2018-02-13 21:29:01 +08:00
0f79c497c5
HBASE-19970 (addendum for 1.x only) Remove unused functions from TableAuthManager.
2018-02-12 16:31:09 -08:00
e6ce789b6f
HBASE-19970 Remove unused functions from TableAuthManager.
...
Functions deleted: setTableUserPermissions, setTableGroupPermissions, setNamespaceUserPermissions,
setNamespaceGroupPermissions, writeTableToZooKeeper, writeNamespaceToZooKeeper
2018-02-12 16:20:36 -08:00
7ced28f099
HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink
2018-02-11 14:58:36 +08:00
0cbb19201c
HBASE-19900 Region-level exception destroy the result of batch
2018-02-07 08:30:55 +08:00
5fa64fb72e
HBASE-19886 Display maintenance mode in shell, web UI
2018-02-06 15:59:44 -08:00
92f733db4b
HBASE-19934 HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-06 11:19:22 -08:00
673e809250
HBASE-19528 - Major Compaction Tool
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-06 05:39:25 -08:00
bdeab93196
HBASE-19905 ReplicationSyncUp tool will not exit if a peer replication is disabled
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-02-04 18:24:32 +05:30
f35bcd2fe3
HBASE-19858 Backport HBASE-14061 (Support CF-level Storage Policy) to branch-1
...
HBASE-14061 Support CF-level Storage Policy
HBASE-14061 Support CF-level Storage Policy (addendum)
HBASE-14061 Support CF-level Storage Policy (addendum2)
HBASE-15172 Support setting storage policy in bulkload
HBASE-17538 HDFS.setStoragePolicy() logs errors on local fs
HBASE-18015 Storage class aware block placement for procedure v2 WALs
HBASE-18017 Reduce frequency of setStoragePolicy failure warnings
Default storage policy if not configured cannot be "NONE"
HBASE-19016 Coordinate storage policy property name for table schema and bulkload
Fix checkstyle warnings
Addressed additional review feedback on backport
2018-02-01 17:10:39 -08:00
e1bcf50b10
HBASE-19728 Add lock to filesCompacting in all place.
2018-01-31 17:01:58 +08:00
2c5140e151
HBASE-19756 Master NPE during completed failed proc eviction
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-01-24 16:48:54 -08:00
73ed127039
HBASE-15321 - Ability to open a HRegion from hdfs snapshot.
2018-01-22 12:22:57 -08:00
0f20c5e0fd
HBASE-19163 "Maximum lock count exceeded" from region server's batch processing
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-19 11:24:19 -08:00
779371625b
HBASE-19816 Refresh repliation sinks on UnknownHostException
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-17 16:26:25 -08:00
48025cc84b
HBASE-11409 - Add more flexibility for input directory structure to LoadIncrementalHFiles
2018-01-12 13:35:26 -08:00
b29a138ecc
HBASE-19483 Add proper privilege check for rsgroup commands addendum
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-11 11:00:07 -08:00
0ef6530aa1
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout.
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-11 10:39:42 -08:00
6f1dd258b1
HBASE-19483 Add proper privilege check for rsgroup commands
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-10 02:25:47 -08:00
e6453ab84b
HBASE-19424 Fix NPE in "/metrics" servlet.
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-08 15:09:27 -08:00
fa07c211ac
HBASE-19684 BlockCacheKey toString Performance
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-08 17:06:44 +08:00
b528cd01b7
HBASE-19708 Avoid NPE when the RPC listener's accept channel is closed
2018-01-05 09:58:03 -08:00
b5ec4d381c
HBASE-15580 Tag coprocessor limitedprivate scope to StoreFile.Reader
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-04 14:15:03 +08:00
939e016425
HBASE-19588 Additional jar dependencies needed for mapreduce
...
PerformanceEvaluation
Signed-off-by: Albert Chu <chu11@llnl.gov>
2018-01-03 21:33:25 -06:00
2c30c9bbb6
HBASE-19490 Rare failure in TestRateLimiter
2018-01-04 10:33:46 +08:00
490728ae7f
HBASE-19691 Removes Global(A) requirement for getClusterStatus
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-03 19:15:49 -05:00
fd33b3589e
HBASE-18625 Splitting of region with replica, doesn't update region list in serverHolding. A server crash leads to overlap.
...
Signed-off-by: ramkrishna.s.vasudevan<ramkrishna.s.vasudevan@intel.com>
2018-01-03 12:32:44 -08:00
6faed49ad1
HBASE-19358 Improve the stability of splitting log when do fail over
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-01-02 17:25:49 +08:00
e866e837fb
HBASE-19551 hbck -boundaries doesn't work correctly (Toshihiro Suzuki)
2017-12-30 23:18:12 -08:00
528eb10826
HBASE-19545 Replace getBytes(StandardCharsets.UTF_8) with Bytes.toBytes
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 20:15:01 +08:00
64d2ebf9f3
HBASE-19514 TestJMXListener now uses random port
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2017-12-22 18:51:44 +01:00
d28732fd98
HBASE-19468 FNFE during scans and flushes (Ram)
2017-12-20 17:06:27 +05:30
08e948b188
HBASE-19546 TestMasterReplication.testCyclicReplication2 uses wrong assertion
2017-12-18 19:59:32 +08:00
ff023bcc4b
HBASE-19485 Minor improvement to TestCompactedHFilesDischarger (Ram)
2017-12-14 13:19:55 -08:00
Andrew Purtell
Thiruvel Thirumoolan
Xu Cang
Balazs Meszaros
Zach York
Ashish Singhi
Vasudevan
Huaxiang Sun
Toshihiro Suzuki
Kevin Risden
Sean Busbey
Reid Chan
Pankaj Kumar
Chia-Ping Tsai
Jan Hentschel
Sakthi
Francis Liu
Abhishek Singh Chouhan
Apekshit Sharma
Sergey Soldatov
Ben Lau
Rahul Gidwani
tedyu
Michael Stack
Esteban Gutierrez
chenheng
huzheng
binlijin
Thiruvel Thirumoolan
huaxiangsun
Scott Wilson
Guangxu Cheng
BELUGA BEHR
Rajeshbabu Chintaguntla
Josh Elser
Jingyun Tian
Peter Somogyi
ramkrish86
zhangduo