6862 Commits

Author SHA1 Message Date
zhangduo
2beda62a10 HBASE-20083 Fix findbugs error for ReplicationSyncUp 2018-02-26 22:13:13 +08:00
zhangduo
8c74d177f6 HBASE-20082 Fix findbugs errors only on master which are introduced by HBASE-19397 2018-02-26 15:50:51 +08:00
Michael Stack
549a6d93d4 HBASE-20043 ITBLL fails against hadoop3
Fix MoveRandomRegionOfTableAction. It depended on old AM behavior.
Make it do explicit move as is required in AMv3; w/o it, it was just
closing region causing test to fail.

Fix pom so hadoop3 profile specifies a different netty3 version.

Bunch of logging format change that came of trying trying to read
the spew from this test.
2018-02-24 17:29:54 -08:00
zhangduo
ba5fb53d14 HBASE-20065 Revisit the timestamp usage in MetaTableAccessor 2018-02-25 09:13:56 +08:00
Sean Busbey
f90b47e3b0 HBASE-20061 HStore synchronized member variable filesCompacting should be private
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-24 10:57:01 -06:00
tedyu
9a7645360b HBASE-20054 Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException 2018-02-23 19:14:52 -08:00
Sean Busbey
216d2d4648 Revert "HBASE-19835 Use explicit casting to avoid side effects"
This reverts commit f1a81618fdd6318df5edded64fbb07e085e10853.

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java

patch reverted changes that happened in parallel without explanation. see jira.
2018-02-23 16:50:15 -06:00
zhangduo
ad5cd50dfc HBASE-20048 Revert serial replication feature 2018-02-23 13:58:31 +08:00
Toshihiro Suzuki
1bc996aa50 HBASE-20049 Region replicas of SPLIT and MERGED regions are kept in in-memory states until restarting master
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-22 20:06:21 -08:00
Apekshit Sharma
51f7b75f1f HBASE-19401 Add missing security checks in RSRpcServices 2018-02-22 16:09:53 -08:00
tedyu
d68f697f39 HBASE-20052 TestRegionOpen#testNonExistentRegionReplica fails due to NPE 2018-02-22 12:35:12 -08:00
Umesh Agashe
8afa49f5d0 HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 10:10:21 -08:00
Michael Stack
3434e99e6c HBASE-20044 TestClientClusterStatus is flakey"
Revert "Revert "HBASE-2004 TestClientClusterStatus is flakey""
This is a revert of a revert, i.e. a reapplication, just so I can fix
the JIRA number.

This reverts commit 6796b8e21faf79e6a738be62f1c7e805e1e88ae3.
2018-02-22 09:44:04 -08:00
Michael Stack
3d2e9e2125 Revert "HBASE-2004 TestClientClusterStatus is flakey"
Revert to fix JIRA number; should be 20044.

This reverts commit 92bb4db9ef367039029017aafe61c51f2c0181f9.
2018-02-22 09:44:04 -08:00
Michael Stack
11af33af2a HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory; ADDEDNDUM to fix TestRegionOpen failure 2018-02-22 09:15:16 -08:00
Josh Elser
4cf846d085 HBASE-20035 Stabilize the flaky TestQuotaStatusRPCs
The test will fail if the quota moves to violation before
the second half of the test.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 09:40:20 -05:00
Michael Stack
2440f807bf Revert "HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs"
Applied prematurely.

This reverts commit 61b55166bf7fe9edc4e8105f217463ed6e693d17.
2018-02-21 18:02:25 -08:00
zhangduo
b328807d25 HBASE-20038 TestLockProcedure.testTimeout is flakey 2018-02-21 17:24:19 -08:00
Esteban Gutierrez
a0900857c7 HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory 2018-02-21 15:56:36 -08:00
Michael Stack
13223c217c HBASE-20042 TestRegionServerAbort flakey; ADDENDUM, RETRY 2018-02-21 15:26:21 -08:00
Umesh Agashe
61b55166bf HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.
2018-02-21 15:14:24 -08:00
Michael Stack
3a3994223c HBASE-2004 TestClientClusterStatus is flakey 2018-02-21 14:53:31 -08:00
Michael Stack
3f82098d4b HBASE-20042 TestRegionServerAbort flakey 2018-02-21 13:26:10 -08:00
Andrew Purtell
173a5bf1f1 HBASE-20027 Add test TestClusterPortAssignment
LocalHBaseCluster forces random port assignment for sake of concurrent unit test
execution friendliness, but we still need a positive test for RPC and info port
assignment.
2018-02-21 13:12:30 -08:00
tedyu
bf5f034463 HBASE-19166 AsyncProtobufLogWriter persists ProtobufLogWriter as class name for backward compatibility 2018-02-21 13:04:59 -08:00
Mike Drob
5d994a24fc HBASE-20039 MR tests out to hbase-mapreduce mobile 2018-02-21 14:54:30 -06:00
Josh Elser
d9b8dcc1d3 HBASE-19953 Ensure post DDL hooks are only called after successful operations
The 1.x functionality of Master DDL operations is that "post" observer hooks
are only invoked when the DDL action was successful. With the async-ness of
ProcV2, we find ourselves in a case where the post-hook may be invoked before
the Procedure runs and fails. We need to introduce some blocking to wait and
see if the Procedure is going to fail on a precondition before invoking the hook.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-21 14:02:49 -05:00
zhangduo
a27ef55a40
HBASE-20037 Race when calling SequenceIdAccounting.resetHighest 2018-02-21 10:05:02 -08:00
anastas
92d04d5751 HBASE-19506: The CellChunkMap index chunks are usually small, so in order to prevent memory underutilization, HBASE-19506 presents small chunks preallocated in a small pool 2018-02-21 10:12:25 +02:00
Chia-Ping Tsai
79d9403a79 HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-20 16:59:48 -08:00
Ted Yu
b75a8f4047 HBASE-19954 Separate TestBlockReorder into individual tests to avoid ShutdownHook suppression error against hadoop3 2018-02-20 14:56:20 -08:00
Apekshit Sharma
6b84393661 HBASE-19400 Add missing security checks in MasterRpcServices
- Added ADMIN permission check for following rpc calls:
  normalize, setNormalizerRunning, runCatalogScan, enableCatalogJanitor, runCleanerChore,
  setCleanerChoreRunning, execMasterService, execProcedure, execProcedureWithRet
- Moved authorizationEnabled check to start of AccessChecker's functions. Currently, and IDK why,
  we call authManager.authorize() first and then discard its result if authorizationEnabled is false. Weird.
2018-02-20 13:37:20 -07:00
Michael Stack
148356a7bf HBASE-20029 @Ignore TestQuotaThrottle and TestReplicasClient#testCancelOfMultiGet 2018-02-20 12:09:47 -08:00
Michael Stack
51cea3e2c3 HBASE-20024 TestMergeTableRegionsProcedure is STILL flakey 2018-02-20 11:08:27 -08:00
zhangduo
69d2becc73 HBASE-20021 TestFromClientSideWithCoprocessor is flakey 2018-02-20 06:24:42 -08:00
zhangduo
b7685307e4 HBASE-20020 Make sure we throw DoNotRetryIOException when ConnectionImplementation is closed 2018-02-20 06:12:13 -08:00
zhangduo
391790ddb0 HBASE-19978 The keepalive logic is incomplete in ProcedureExecutor 2018-02-19 17:13:47 -08:00
Reid Chan
a9a6eed372 HBASE-19950 Introduce a ColumnValueFilter
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-20 04:56:13 +08:00
eshcar
f3bb9b9613 HBASE-18294 Reduce global heap pressure: flush based on heap occupancy 2018-02-18 09:55:44 +02:00
Michael Stack
f3ff55a2b4 HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey 2018-02-17 11:38:26 -08:00
Michael Stack
68d509bc1f HBASE-20013 TestZKPermissionWatcher is flakey 2018-02-16 23:22:23 -08:00
Chia-Ping Tsai
dad90f6cce HBASE-19680 BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations 2018-02-17 07:33:58 +08:00
Michael Stack
abf7de702c HBASE-20011 Disable TestRestoreSnapshotFromClientWithRegionReplicas; it is flakey. Needs attention. 2018-02-16 14:46:24 -08:00
Mike Drob
8cf7a9d51e HBASE-19920 Lazy init for ProtobufUtil classloader 2018-02-16 09:12:15 -06:00
Umesh Agashe
9497eac4e0 HBASE-19988 Fixed chatty log from HRegion#lockRowsAndBuildMiniBatch() when interrupted while waiting on a row lock 2018-02-16 07:01:56 -08:00
Michael Stack
40f8d20cf7 HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
Remove assert in splittableregionprocedure. It was in the prepare.
Was causing fail in legit case where a region split follows a
table split BEFORE the parent has been GC'd. The region split
finds the parent in SPLIT state which is right. The assert was
having us fail. No need.

Also disabled TestHTrace since not supported in 2.0.0 and flakey.
2018-02-15 20:46:32 -08:00
Michael Stack
50c705dad9 HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService
Only call server.checkIfShouldMoveSystemRegionAsync if a node has been
added. Do not call it if only one regionserver in cluster. Make it
so ServerCrashProcedure runs before it. Add logging if
server.checkIfShouldMoveSystemRegionAsync was responsible for
MOVE (Previous was a mystery when it cut in).

Previous we'd call it when there was a nodeChildrenChanged. These
happen before nodeDeleted. If a server crashed,
checkIfShouldMoveSystemRegionAsync could run first, find the
server that had not yet registered as crashed, find system
tables on it and then try to move them. It would fail because
server would not respond to RPC. The region move would then
be waiting on the servercrashprocedure to wake it up when
done processing but this move had locked the region so
SCP couldn't run....
2018-02-15 06:08:55 -08:00
Thiruvel Thirumoolan
816d860228 HBASE-19996 Some nonce procs might not be cleaned up (follow up HBASE-19756)
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-15 04:47:53 -08:00
anastas
07e93458fe HBASE-19930: Fixing the bug, in the rare case when there is a merge into CCM and the one of the old segments has a big cell allocated on-heap and it needs to be copied to the MSLAB 2018-02-15 14:34:12 +02:00
zhangduo
c18e7a963d HBASE-20000 Remove the quantum logic in FairQueue, always put high priority queue in front 2018-02-15 13:49:54 +08:00