197bd79070
HBASE-20084 Refactor the RSRpcServices#doBatchOp
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:15:34 +08:00
7f6e971c4c
HBASE-20093 Replace ServerLoad by ServerMetrics for ServerManager
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 14:57:50 +08:00
ba063abd2f
HBASE-20100 TestEnableTableProcedure flakey
...
Allow OPEN as a possible state when update region transition state.
Usually state is OPENING but if crash before finish step is completed,
on replay, master may have read that the state is OPEN from meta table
and so will think it open... When we replay the procedure finish, allow
that the region is already OPEN.
2018-02-27 14:51:21 -08:00
f01c90bc0e
HBASE-19614 Use ArrayDeque as Queue instead of LinkedList in CompoundBloomFilterWriter
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-02-27 11:50:52 -08:00
0fa5d69fc7
HBASE-20074 [FindBugs] Same code on both branches in CompactingMemStore#initMemStoreCompactor
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-27 11:21:40 -06:00
e4ce38df32
Revert "for creating patch HBASE-20074-V01.patch"
...
This reverts commit 73028d5bd9
2018-02-27 11:20:54 -06:00
dbd8013057
HBASE-20092 Fix TestRegionMetrics#testRegionMetrics
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-27 16:48:17 +08:00
f06a89b531
HBASE-20066 Region sequence id may go backward after split or merge
2018-02-27 15:33:07 +08:00
d272ac908c
HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review
2018-02-26 23:16:21 -08:00
b11e506664
HBASE-20069 fix existing findbugs errors in hbase-server
2018-02-26 16:01:31 -08:00
73028d5bd9
for creating patch HBASE-20074-V01.patch
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-26 09:57:22 -08:00
7cfb46432f
HBASE-20036 TestAvoidCellReferencesIntoShippedBlocks timed out (Ram)
2018-02-26 22:07:42 +05:30
2beda62a10
HBASE-20083 Fix findbugs error for ReplicationSyncUp
2018-02-26 22:13:13 +08:00
8c74d177f6
HBASE-20082 Fix findbugs errors only on master which are introduced by HBASE-19397
2018-02-26 15:50:51 +08:00
549a6d93d4
HBASE-20043 ITBLL fails against hadoop3
...
Fix MoveRandomRegionOfTableAction. It depended on old AM behavior.
Make it do explicit move as is required in AMv3; w/o it, it was just
closing region causing test to fail.
Fix pom so hadoop3 profile specifies a different netty3 version.
Bunch of logging format change that came of trying trying to read
the spew from this test.
2018-02-24 17:29:54 -08:00
ba5fb53d14
HBASE-20065 Revisit the timestamp usage in MetaTableAccessor
2018-02-25 09:13:56 +08:00
f90b47e3b0
HBASE-20061 HStore synchronized member variable filesCompacting should be private
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-24 10:57:01 -06:00
9a7645360b
HBASE-20054 Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
2018-02-23 19:14:52 -08:00
216d2d4648
Revert "HBASE-19835 Use explicit casting to avoid side effects"
...
This reverts commit f1a81618fd
2018-02-23 16:50:15 -06:00
ad5cd50dfc
HBASE-20048 Revert serial replication feature
2018-02-23 13:58:31 +08:00
1bc996aa50
HBASE-20049 Region replicas of SPLIT and MERGED regions are kept in in-memory states until restarting master
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-22 20:06:21 -08:00
51f7b75f1f
HBASE-19401 Add missing security checks in RSRpcServices
2018-02-22 16:09:53 -08:00
d68f697f39
HBASE-20052 TestRegionOpen#testNonExistentRegionReplica fails due to NPE
2018-02-22 12:35:12 -08:00
8afa49f5d0
HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
...
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 10:10:21 -08:00
3434e99e6c
HBASE-20044 TestClientClusterStatus is flakey"
...
Revert "Revert "HBASE-2004 TestClientClusterStatus is flakey""
This is a revert of a revert, i.e. a reapplication, just so I can fix
the JIRA number.
This reverts commit 6796b8e21f
2018-02-22 09:44:04 -08:00
3d2e9e2125
Revert "HBASE-2004 TestClientClusterStatus is flakey"
...
Revert to fix JIRA number; should be 20044.
This reverts commit 92bb4db9ef
2018-02-22 09:44:04 -08:00
11af33af2a
HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory; ADDEDNDUM to fix TestRegionOpen failure
2018-02-22 09:15:16 -08:00
4cf846d085
HBASE-20035 Stabilize the flaky TestQuotaStatusRPCs
...
The test will fail if the quota moves to violation before
the second half of the test.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 09:40:20 -05:00
2440f807bf
Revert "HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs"
...
Applied prematurely.
This reverts commit 61b55166bf
2018-02-21 18:02:25 -08:00
b328807d25
HBASE-20038 TestLockProcedure.testTimeout is flakey
2018-02-21 17:24:19 -08:00
a0900857c7
HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory
2018-02-21 15:56:36 -08:00
13223c217c
HBASE-20042 TestRegionServerAbort flakey; ADDENDUM, RETRY
2018-02-21 15:26:21 -08:00
61b55166bf
HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
...
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.
2018-02-21 15:14:24 -08:00
3a3994223c
HBASE-2004 TestClientClusterStatus is flakey
2018-02-21 14:53:31 -08:00
3f82098d4b
HBASE-20042 TestRegionServerAbort flakey
2018-02-21 13:26:10 -08:00
173a5bf1f1
HBASE-20027 Add test TestClusterPortAssignment
...
LocalHBaseCluster forces random port assignment for sake of concurrent unit test
execution friendliness, but we still need a positive test for RPC and info port
assignment.
2018-02-21 13:12:30 -08:00
bf5f034463
HBASE-19166 AsyncProtobufLogWriter persists ProtobufLogWriter as class name for backward compatibility
2018-02-21 13:04:59 -08:00
5d994a24fc
HBASE-20039 MR tests out to hbase-mapreduce mobile
2018-02-21 14:54:30 -06:00
d9b8dcc1d3
HBASE-19953 Ensure post DDL hooks are only called after successful operations
...
The 1.x functionality of Master DDL operations is that "post" observer hooks
are only invoked when the DDL action was successful. With the async-ness of
ProcV2, we find ourselves in a case where the post-hook may be invoked before
the Procedure runs and fails. We need to introduce some blocking to wait and
see if the Procedure is going to fail on a precondition before invoking the hook.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-21 14:02:49 -05:00
a27ef55a40
HBASE-20037 Race when calling SequenceIdAccounting.resetHighest
2018-02-21 10:05:02 -08:00
92d04d5751
HBASE-19506: The CellChunkMap index chunks are usually small, so in order to prevent memory underutilization, HBASE-19506 presents small chunks preallocated in a small pool
2018-02-21 10:12:25 +02:00
79d9403a79
HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-20 16:59:48 -08:00
b75a8f4047
HBASE-19954 Separate TestBlockReorder into individual tests to avoid ShutdownHook suppression error against hadoop3
2018-02-20 14:56:20 -08:00
6b84393661
HBASE-19400 Add missing security checks in MasterRpcServices
...
- Added ADMIN permission check for following rpc calls:
normalize, setNormalizerRunning, runCatalogScan, enableCatalogJanitor, runCleanerChore,
setCleanerChoreRunning, execMasterService, execProcedure, execProcedureWithRet
- Moved authorizationEnabled check to start of AccessChecker's functions. Currently, and IDK why,
we call authManager.authorize() first and then discard its result if authorizationEnabled is false. Weird.
2018-02-20 13:37:20 -07:00
148356a7bf
HBASE-20029 @Ignore TestQuotaThrottle and TestReplicasClient#testCancelOfMultiGet
2018-02-20 12:09:47 -08:00
51cea3e2c3
HBASE-20024 TestMergeTableRegionsProcedure is STILL flakey
2018-02-20 11:08:27 -08:00
69d2becc73
HBASE-20021 TestFromClientSideWithCoprocessor is flakey
2018-02-20 06:24:42 -08:00
b7685307e4
HBASE-20020 Make sure we throw DoNotRetryIOException when ConnectionImplementation is closed
2018-02-20 06:12:13 -08:00
391790ddb0
HBASE-19978 The keepalive logic is incomplete in ProcedureExecutor
2018-02-19 17:13:47 -08:00
a9a6eed372
HBASE-19950 Introduce a ColumnValueFilter
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-20 04:56:13 +08:00
f3bb9b9613
HBASE-18294 Reduce global heap pressure: flush based on heap occupancy
2018-02-18 09:55:44 +02:00
f3ff55a2b4
HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey
2018-02-17 11:38:26 -08:00
68d509bc1f
HBASE-20013 TestZKPermissionWatcher is flakey
2018-02-16 23:22:23 -08:00
dad90f6cce
HBASE-19680 BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations
2018-02-17 07:33:58 +08:00
abf7de702c
HBASE-20011 Disable TestRestoreSnapshotFromClientWithRegionReplicas; it is flakey. Needs attention.
2018-02-16 14:46:24 -08:00
8cf7a9d51e
HBASE-19920 Lazy init for ProtobufUtil classloader
2018-02-16 09:12:15 -06:00
9497eac4e0
HBASE-19988 Fixed chatty log from HRegion#lockRowsAndBuildMiniBatch() when interrupted while waiting on a row lock
2018-02-16 07:01:56 -08:00
40f8d20cf7
HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
...
Remove assert in splittableregionprocedure. It was in the prepare.
Was causing fail in legit case where a region split follows a
table split BEFORE the parent has been GC'd. The region split
finds the parent in SPLIT state which is right. The assert was
having us fail. No need.
Also disabled TestHTrace since not supported in 2.0.0 and flakey.
2018-02-15 20:46:32 -08:00
50c705dad9
HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService
...
Only call server.checkIfShouldMoveSystemRegionAsync if a node has been
added. Do not call it if only one regionserver in cluster. Make it
so ServerCrashProcedure runs before it. Add logging if
server.checkIfShouldMoveSystemRegionAsync was responsible for
MOVE (Previous was a mystery when it cut in).
Previous we'd call it when there was a nodeChildrenChanged. These
happen before nodeDeleted. If a server crashed,
checkIfShouldMoveSystemRegionAsync could run first, find the
server that had not yet registered as crashed, find system
tables on it and then try to move them. It would fail because
server would not respond to RPC. The region move would then
be waiting on the servercrashprocedure to wake it up when
done processing but this move had locked the region so
SCP couldn't run....
2018-02-15 06:08:55 -08:00
816d860228
HBASE-19996 Some nonce procs might not be cleaned up (follow up HBASE-19756)
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-15 04:47:53 -08:00
07e93458fe
HBASE-19930: Fixing the bug, in the rare case when there is a merge into CCM and the one of the old segments has a big cell allocated on-heap and it needs to be copied to the MSLAB
2018-02-15 14:34:12 +02:00
c18e7a963d
HBASE-20000 Remove the quantum logic in FairQueue, always put high priority queue in front
2018-02-15 13:49:54 +08:00
d0f2d18ca7
HBASE-19980 NullPointerException when restoring a snapshot after splitting a region
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-14 09:37:16 -08:00
8d26736bc2
HBASE-19116 Currently the tail of hfiles with CellComparator* classname makes it so hbase1 can't open hbase2 written hfiles; fix
...
Serializing, if appropriate, write the hbase-1.x version of the
Comparator to the hfile trailer so hbase-1.x files can read hbase-2.x
hfiles (they are the same format).
2018-02-14 07:46:30 -08:00
2f1b3eab67
HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService
2018-02-13 22:32:20 -08:00
6e35f5eab9
HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI AGAIN (make TestAsyncTableAdminAPI3... already an TestAsyncTableAdminAPI2)
2018-02-13 21:06:00 -08:00
39e191e559
HBASE-19979 ReplicationSyncUp tool may leak Zookeeper connection
2018-02-13 08:59:02 -08:00
ba402b1e7b
Revert "HBASE-19970 Remove unused functions from TableAuthManager."
...
This reverts commit 7cc239fb5a
2018-02-13 06:19:08 -08:00
2f48fdbb26
HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
2018-02-13 21:08:59 +08:00
16f1f5b494
HBASE-19977 FileMmapEngine allocation of byte buffers should be
...
synchronized (Ram)
2018-02-13 15:49:37 +05:30
b4622ffad7
HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI
2018-02-12 22:52:56 -08:00
cf57ea15f1
HBASE-19965 Fix flaky TestAsyncRegionAdminApi
2018-02-12 22:27:12 -08:00
7cc239fb5a
HBASE-19970 Remove unused functions from TableAuthManager.
...
Functions deleted: setTableUserPermissions, setTableGroupPermissions, setNamespaceUserPermissions,
setNamespaceGroupPermissions, writeTableToZooKeeper, writeNamespaceToZooKeeper
2018-02-12 16:01:29 -08:00
c2ee82c909
HBASE-19986 If HBaseTestClassRule timesout a test, thread dump
2018-02-12 15:28:40 -08:00
00f8877323
HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink; ADDENDUM to fix TestReplicationSink
2018-02-12 10:47:25 -08:00
67b69fb2c7
HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
...
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.
The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.
D hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
Unused.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.
A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
Test migration and mirroring.
2018-02-12 08:47:02 -08:00
8ff783f76d
HBASE-19554 Dump thread info before failure
2018-02-12 16:03:52 +08:00
629eaf8b74
HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink
2018-02-11 14:48:21 +08:00
0593dda663
HBASE-19951 Cleanup the explicit timeout value for test method
2018-02-10 09:24:31 -08:00
32f235abee
HBASE-19966 The WriteEntry for WALKey maybe null if we failed to call WAL.append
2018-02-09 19:09:40 -08:00
2b63af376e
HBASE-19923 Reset peer state and config when refresh replication source failed
2018-02-09 18:54:03 +08:00
0a5f26324c
HBASE-19942 (addendum) Fix flaky TestSimpleRpcScheduler
2018-02-09 13:37:06 +08:00
f1a81618fd
HBASE-19835 Use explicit casting to avoid side effects
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-08 12:32:36 -08:00
277ce3d8a3
HBASE-19956 Remove category as a consideration timing out tests; set all test to timeout at 10minutes regardless
2018-02-07 23:10:12 -08:00
f5dbdf0dab
HBASE-19947 ITU should overwrite HTU local FS
2018-02-07 16:53:54 -06:00
dcbb331792
HBASE-19929 Call RS.stop on a session expired RS may hang
2018-02-07 15:52:04 +08:00
d8b999e695
HBASE-19900 Region-level exception destroy the result of batch
2018-02-07 08:41:56 +08:00
a5b86dd77a
HBASE-19942 Fix flaky TestSimpleRpcScheduler
2018-02-07 07:54:39 +08:00
380083e7d3
HBASE-19886 Display maintenance mode in shell, web UI
2018-02-06 14:53:32 -08:00
b5ccfecc96
HBASE-19946 More test clean up for local-mode HTU
2018-02-06 15:20:39 -06:00
001f9cc5ea
HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: yet more debug
2018-02-06 09:47:30 -08:00
590bee78dc
HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: more debug
2018-02-06 06:35:36 -08:00
77153d4dc6
HBASE-19944 Fix timeout TestVisibilityLabelsWithCustomVisLabService
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-02-06 20:30:08 +08:00
3fa27c2382
HBASE-19941 Flaky TestCreateTableProcedure times out in nightly, needs to LargeTests
2018-02-05 22:13:53 -08:00
11e3a83870
HBASE-19934 HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting (Toshihiro Suzuki)
2018-02-05 21:03:56 -08:00
3bb8daa605
HBASE-19939 Fixed NPE in tests TestSplitTableRegionProcedure#testSplitWithoutPONR() and testRecoveryAndDoubleExecution()
...
Value of 'htd' is null as it is initialized in the constructor but when the object is deserialized its null. Got rid of member variable htd and made it local to method.
2018-02-05 20:48:56 -08:00
f5197979aa
HBASE-19927 Addendum join on RegionServerThread instead of HRegionServer
2018-02-06 10:01:23 +08:00
6307689a6e
HBASE-19940 TestMetaShutdownHandler flakey
2018-02-05 12:37:48 -08:00
9f2149f171
HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM Adding debug
2018-02-05 11:01:09 -08:00
c245bd5c03
HBASE-19837 Flakey TestRegionLoad; ADDENDUM Report more often and wait less time on change (also add some debug on TestMetaShutdown test)
2018-02-05 08:40:26 -08:00
c5f86f2ce4
HBASE-19703 Functionality added as part of HBASE-12583 is not working after moving the split code to master
...
Co-authored-by: Michael Stack <stack@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-05 23:56:18 +08:00
ad3a1ba495
HBASE-19936 Introduce a new base class for replication peer procedure
2018-02-05 20:23:19 +08:00
88d6e06a1f
HBASE-19554 reenable TestDLSAsyncFSWAL/TestDLSFSHLog for debugging
2018-02-05 08:58:16 +08:00
e1cd10b002
HBASE-19927 TestFullLogReconstruction flakey
2018-02-05 08:41:32 +08:00
ab5a26ad5e
HBASE-19931 TestMetaWithReplicas failing 100% of the time in testHBaseFsckWithMetaReplicas
2018-02-04 14:38:21 -08:00
170ffbba68
HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress
2018-02-04 14:59:10 +02:00
14420e1b41
HBASE-19926 Use a separated class to implement the WALActionListener for Replication
2018-02-04 20:36:43 +08:00
397d34736e
HBASE-19905 ReplicationSyncUp tool will not exit if a peer replication is disabled
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-02-04 17:52:38 +05:30
2e1ec3d3d8
HBASE-19914 Refactor TestVisibilityLabelsOnNewVersionBehaviorTable
2018-02-04 13:27:28 +08:00
bfd74686c7
HBASE-19928 TestVisibilityLabelsOnNewVersionBehaviorTable fails
2018-02-03 10:17:40 -08:00
12f3c82a86
HBASE-19919 Tidying up logging; ADDENDUM Fix tests w/ mocked Servers
2018-02-03 09:25:16 -08:00
06dec20582
HBASE-19919 Tidying up logging
2018-02-03 08:42:02 -08:00
6519b98ac3
Revert "HBASE-19919 Tidying up logging"
...
This reverts commit 40250f8c5f
2018-02-03 18:18:09 +08:00
40250f8c5f
HBASE-19919 Tidying up logging
2018-02-02 22:52:41 -08:00
41974efa85
HBASE-19726 Failed to start HMaster due to infinite retrying on meta assign
2018-02-02 18:29:33 -08:00
ad580acc89
HBASE-19918 Promote TestAsyncClusterAdminApi to LargeTests
2018-02-03 09:39:40 +08:00
2f4d0b94bc
HBASE-19720 Rename WALKey#getTabnename to WALKey#getTableName
2018-02-02 19:43:26 +08:00
fc6d140adf
HBASE-19904 Break dependency of WAL constructor on Replication
2018-02-02 14:10:29 +08:00
a11258599e
HBASE-19895 Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks (Ankit Singhal)
2018-02-01 18:27:18 -08:00
99b9fff07b
HBASE-19841 Every HTU should be local until DFS starts
2018-02-01 20:24:11 -06:00
94dad844af
HBASE-19913 Split TestStochasticLoadBalancer2
2018-02-02 08:29:28 +08:00
3622bb0333
HBASE-19147 TestCacheOnWrite Times Out
2018-02-01 16:14:51 -08:00
adccbb7edf
HBASE-19897 RowMutations should follow the fluent pattern
2018-02-02 05:33:08 +08:00
d472422475
HBASE-19884 BucketEntryGroup's equals, hashCode and compareTo methods are not consistent
...
Move back to default equals and hashCode.
Remove compareTo and Comparator to PriorityQueue.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 13:21:58 -08:00
38c8144a06
HBASE-19912 Remove useless 'writeToWAL' flag of Region#checkAndRowMutate
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:33:48 -08:00
57911d02c6
HBASE-19839 Fixed flakey tests TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution
...
* Added a comment in MergeTableRegionsProcedure and SplitTableRegionProcedure explaining specific rollbacks has side effect that AssignProcedure/s are submitted asynchronously and those procedures may continue to execute even after rollback() is done.
* Updated comments in tests with correct rollback state to abort
* Added overloaded method MasterProcedureTestingUtility#testRollbackAndDoubleExecution which takes additional argument for waiting on all procedures to finish before asserting conditions
* Updated TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution to use newly added method
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:01:30 -08:00
b21b8bfb91
HBASE-19906 TestZooKeeper Timeout Includes move of TestQoSFunction from smalltests to mediumtests; it timedout in a run.
2018-02-01 10:53:18 -08:00
9272f40a5c
HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus; ADDENDUM changes TestCheckTestClasses to be medium sized test
2018-01-31 20:27:34 -08:00
b9cb1187b5
HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus
2018-01-31 20:23:14 -08:00
e17529ba73
HBASE-19910 TestBucketCache TimesOut
2018-01-31 18:17:01 -08:00
28ee4be0fe
HBASE-19909 TestRegionLocationFinder Timeout
2018-01-31 17:55:16 -08:00
bc0e9ce8f4
HBASE-19908 TestCoprocessorShortCircuitRPC Timeout....
2018-01-31 17:32:39 -08:00
414b2d0889
HBASE-19907 TestMetaWithReplicas still flakey
2018-01-31 15:17:38 -08:00
60827fc1ea
HBASE-19528 Major Compaction Tool; ADDENDUM
...
Add @ClassRule to new tests and rename those added here so they
align w/ how we name other Test classes.
2018-01-31 14:18:24 -08:00
4b3b627abe
HBASE-19528 - Major Compaction Tool
2018-01-31 10:18:03 -08:00
7c318cead9
HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules
2018-01-31 20:49:18 +08:00
f7faad0477
HBASE-19728 Add lock to filesCompacting in all place.
2018-01-31 14:05:30 +08:00
194890b694
HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey
...
Recategorize TestCoprocessorWhitelistMasterObserver as Large test
2018-01-30 06:01:24 -08:00
98c9d8f7a1
HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey
2018-01-29 21:35:28 -08:00
34c6c99041
HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2) - addendum
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-29 14:03:08 -08:00
0b9a0dc951
HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM to fix Checksyte
2018-01-29 09:36:30 -08:00
4f547b3817
HBASE-19840 Flakey TestMetaWithReplicas
...
Fix two issues:
# Meta Replicas can all be assigned to the same server. This
will call the test to hang when we do our kill of the server
hosting meta because there'll be no replicas to read from
as test intends. Check is to look for this condition on
startup and adjust if we come across it. Replicas cross-cut
assignment. They need work.
# Other issue was shutdown. The master started toward the
end of the test may not have come up fully by the time
shutdown is called. We could be stuck assigning the
meta replicas. Have shutdown shutdown the procedure
executor engine.
There is other cleanup and notes in the below.
M HMaster
Remove the silly stops in startup now we have real
means of shutting down Master during init.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
This replica stuff was doing stuff it shouldn't be doing
like setting core Master state flags. It may have made
sense once but now meta is assigned by a Pv2 Procedure
so the flag setting in here is meddlesome. Clear out
methods no longer needed.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Remove unused methods.
Changes local variable names so they align w/ our naming elsewhere in
code base.
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
Check for all replicas on the one server.
2018-01-29 09:21:30 -08:00
dab51c7900
HBASE-19885 Promote TestAssignmentManager to LargeTests
2018-01-30 00:24:45 +08:00
5b93ea1556
HBASE-19866 TestRegionServerReportForDuty doesn't timeout
...
Synchronized on authManager instead of RpcServer
2018-01-29 21:23:47 +08:00
305da1098a
HBASE-19882 Promote TestProcedureManager to MediumTests
2018-01-29 17:48:44 +08:00
f9480a56c7
HBASE-19881 Promote TestRegionReplicaReplicationEndpoint to LargeTests
2018-01-29 16:15:45 +08:00
94dfc7d60a
HBASE-19880 Promote TestFuzzyRowFilterEndToEnd to LargeTests
2018-01-29 16:15:18 +08:00
80438924ff
HBASE-19879 Promote TestAcidGuaranteesXXX to LargeTests
2018-01-29 14:55:04 +08:00
918599ef12
HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs
2018-01-29 08:43:56 +08:00
851e17987d
HBASE-19869 TestLockManager now uses the right class for logging
2018-01-28 13:54:57 +01:00
f1502a3aca
HBASE-19867 Split TestStochasticLoadBalancer into several small tests
2018-01-27 18:25:46 +08:00
a5a8c4f3f2
Revert "HBASE-19841 LocalHTU to not enforce stream capabilities"
...
At Mike Drobs' request!
2018-01-26 15:11:50 -08:00
9ea152d235
HBASE-19841 LocalHTU to not enforce stream capabilities
2018-01-26 10:50:16 -06:00
94a1c051ba
HBASE-19861 - Avoid using RPCs when querying table infos for master status pages
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-26 08:02:14 -08:00
39b912aae9
HBASE-19818 Scan time limit not work if the filter always filter row key
2018-01-26 17:02:22 +08:00
01c34243fe
HBASE-19866 Add cateogry based timeout using Timeout Rule to TestRegionServerReportForDuty.
2018-01-26 00:30:56 -08:00
1beb687f4e
HBASE-19862 addendum: Checkstyle fixes, remove unused imports and variables. Add @Rule for timeout.
2018-01-26 00:24:35 -08:00
61358985b1
HBASE-19862 Fix TestTokenAuthentication - mock RegionCoprocessorEnvironment should be of type HasRegionServerServices also.
2018-01-25 23:15:45 -08:00
ce50830a0a
HBASE-19756 Master NPE during completed failed proc eviction
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-01-24 16:42:58 -08:00
970636c5af
HBASE-19838 Can not shutdown backup master cleanly when it has already tried to become the active master
...
On Master@shutdown, close the shared Master connection to kill any
ongoing RPCs by hosted clients.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Call close ont the Master shared clusterconnection to kill any ongoing
rpcs.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Remove guts of close; we were closing the Masters connection....not
our responsibility.
Added unit test written by Duo Zhang which demonstrates the case where
Master will not go down.
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-23 11:07:00 +08:00
541f8ad8a8
HBASE-4224 Need a flush by regionserver rather than by table option
2018-01-23 09:51:05 +08:00
93a182f286
HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2)
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-22 17:20:59 -08:00
d49357f265
HBASE-19840 Flakey TestMetaWithReplicas
...
Adding debuging
2018-01-22 12:28:56 -08:00
b577fa0814
HBASE-15321 - Ability to open a HRegion from hdfs snapshot.
2018-01-22 12:09:38 -08:00
be496ba8ea
HBASE-19774 incorrect behavior of locateRegionInMeta
2018-01-22 11:48:55 -08:00
c60359959c
HBASE-19837 Flakey TestRegionLoad
...
Add debug, change name of the rows so readable.
2018-01-22 08:01:57 -08:00
86f4df5f74
HBASE-19836 Fix TestZooKeeper.testLogSplittingAfterMasterRecoveryDueToZKExpiry
2018-01-22 18:32:31 +08:00
f3c563fc00
HBASE-19837 Flakey TestRegionLoad
...
Add debug, change name of the rows so readable.
2018-01-21 23:24:23 -08:00
27d00f5861
HBASE-19794 TestZooKeeper hangs
...
Kill backup master first
Add some cleanup around NamespaceManager
Shorten the timeout waiting on namespace manager as workaround
until we have better soln for interrupting ongoing client rpcs.
Do it in general for all tests.
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-21 20:59:16 +08:00
7fe4aa6fe4
HBASE-19828 Flakey TestRegionsOnMasterOptions.testRegionsOnAllServers
...
Rename the PE Worker threads.
Send an interrupt if worker taking a long time to go down
(it may be RPC'ing out to a dead server, retrying so
interrupt). Also join on the ProcedureExecutor shutting down.
This will make problems shutting down more obvious.
Disable TestRegionsOnMasterOptions. Master carrying Regions is broke.
2018-01-19 21:54:19 -08:00
11d6e6b1e6
HBASE-19829 hadoop-minicluster pulls zookeeper:test-jar:tests 3.4.6
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-19 15:18:22 -08:00
579586c697
HBASE-19827 Addendum for Flakey TestAssignmentManager Don't clear old location from RTP...
2018-01-19 13:46:44 -08:00
c66b42fd9d
Revert "HBASE-19827 Addendum for Flakey TestAssignmentManager"
...
Premature commit.
This reverts commit fa66c7e28e
2018-01-19 11:47:28 -08:00
298dc0c626
HBASE-19822 Override cancel(boolean) in LogCleaner (Addendum)
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-19 11:46:46 -08:00
fa66c7e28e
HBASE-19827 Addendum for Flakey TestAssignmentManager
...
Don't clear old location from RTP...
2018-01-19 10:59:47 -08:00
8b520ce50d
HBASE-19822 HFileCleaner threads stick around after shutdown stuck on queue#take Interrupt wait on queue when cancel is called.
2018-01-18 13:52:51 -08:00
cde1f821eb
HBASE-19799 Add web UI to rsgroup
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-18 12:49:43 -08:00
581fabe7b2
HBASE-19815 Flakey TestAssignmentManager.testAssignWithRandExec (Part Two).
...
Part One cleaned up a ClassCastException.
Part Two adds the ServerCrashProcedure#handleRIT behavior to RecoverMetaProcedure.
Adds debug in the test.
2018-01-18 11:32:21 -08:00
646770dd51
HBASE-19527 Make ExecutorService threads daemon=true
...
Set the ProcedureExcecutor worker threads as daemon.
Ditto for the timeout thread.
Remove hack from TestRegionsOnMasterOptions that was
put in place because the test would not go down.
2018-01-18 11:30:15 -08:00
b3eff31432
HBASE-19821 TestCleanerChore#testOnConfigurationChange() requires at least 4 processors to get passed
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-18 11:20:29 -08:00
09ffbb5b68
HBASE-19820 Restore public constructor of MiniHBaseCluster (API compat).
2018-01-18 09:05:30 -08:00
7573e07cc6
HBASE-19806 Lower max versions for table column family of hbase:meta
2018-01-18 07:57:22 -08:00
678b98a726
HBASE-19812 TestFlushSnapshotFromClient fails because of failing region.flush
2018-01-18 16:23:07 +08:00
e3d15834cb
HBASE-19816 Refresh repliation sinks on UnknownHostException
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-17 16:24:48 -08:00
eeb40ff66c
Revert "HBASE-19791 TestZKAsyncRegistry hangs"
...
Premature push
This reverts commit d3a306d81d
2018-01-17 15:19:48 -08:00
53d0c2388d
HBASE-19815 Flakey TestAssignmentManager.testAssignWithRandExec
2018-01-17 13:52:39 -08:00
a3c98b2dd8
HBASE-19808 Reenable TestMultiParallel
2018-01-17 08:58:37 -08:00
8a8be33c71
HBASE-19736 Remove BaseLogCleanerDelegate deprecated #isLogDeletable(FileStatus) and use #isFileDeletable(FileStatus) instead
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-17 21:16:23 +08:00
d8d6ecdad1
HBASE-19792 TestReplicationSmallTests.testDisableEnable fails
2018-01-17 14:23:05 +08:00
d3a306d81d
HBASE-19791 TestZKAsyncRegistry hangs
2018-01-16 12:34:05 -08:00
d155a4e33d
HBASE-19732 Replica regions does not return back the MSLAB chunks to pool
...
(Ram)
2018-01-15 22:20:27 +05:30
3a66f15557
HBASE-19797 Operator priority leads to wrong logic in ReplicationSourceWALReader
2018-01-15 18:10:50 +08:00
3b638f7ea1
HBASE-19793 Minor improvements on Master/RS startup
2018-01-15 17:53:28 +08:00
e546034f09
HBASE-19795 Move the tests which only need zookeeper in TestZooKeeper to hbase-zookeeper module
2018-01-15 10:57:51 +08:00
c554340a91
HBASE-19768 RegionServer startup failing when DN is dead
2018-01-14 17:30:50 +08:00
4ddfecac56
HBASE-19787 Fix or disable tests broken in branch-2 so can cut beta-1
...
M dev-support/make_rc.sh
Disable checkstyle building site. Its an issue being fixed over in HBASE-19780
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
The clusterid was being set into the process only after the
regionserver registers with the Master. That can be too late for some
test clients in particular. e.g. TestZKAsyncRegistry needs it as soon
as it goes to run which could be before Master had called its run
method which is regionserver run method which then calls back to the
master to register itself... and only then do we set the clusterid.
HBASE-19694 changed start order which made it so this test failed.
Setting the clusterid right after we set it in zk makes the test pass.
Another change was that backup masters were not going down on stop.
Backup masters were sleeping for the default zk period which is 90
seconds. They were not being woken up to check for stop. On stop
master now tells active master manager.
M hbase-server/src/test/java/org/apache/hadoop/hbase/TestJMXConnectorServer.java
Prevent creation of acl table. Messes up our being able to go down
promptly.
M hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReadRequestMetrics.java
Disabled for now because it wants to run with regions on the Master...
currently broke!
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestZKAsyncRegistry.java
Add a bit of debugging.
M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSAsyncFSWAL.java
Disabled. Fails 40% of the time.
M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSFSHLog.java
Disabled. Fails 33% of the time.
Disabled stochastic load balancer for favored nodes because it fails on
occasion and we are not doing favored nodes in branch-2.
2018-01-12 14:09:56 -08:00
c799b0e69b
HBASE-19694 The initialization order for a fresh cluster is incorrect Addendum. Adds in a missing line that helps debugging.
2018-01-12 13:09:05 -08:00
20ccaef841
HBASE-19783 Change replication peer cluster key/endpoint from a not-null value to null is not allowed
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-12 21:41:51 +08:00
4bd6ac3e10
HBASE-19746 Add default impl to Cell#getType
2018-01-11 15:03:20 -08:00
3787c60cd1
HBASE-19694 The initialization order for a fresh cluster is
...
incorrect; ADDENDUM Fix compile issue.
2018-01-11 14:37:18 -08:00
d0aff98a67
HBASE-19694 The initialization order for a fresh cluster is incorrect
...
Become active Master before calling the super class's run method. Have
the wait-on-becoming-active-Master be in-line rather than off in a
background thread (i.e. undo running thread in startActiveMasterManager)
Purge the fragile HBASE-16367 hackery that attempted to fix this issue
previously by adding a latch to try and hold up superclass RegionServer
until cluster id set by subclass Master.
2018-01-11 14:29:11 -08:00
e0eea94c98
HBASE-19758 Split TestHCM to several smaller tests
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-11 11:39:51 -08:00
200c3d3b8e
HBASE-11409 - Add more flexibility for input directory structure to LoadIncrementalHFiles
2018-01-11 11:05:11 -08:00
9f69be0011
Revert due to missing JIRA ticket - "Add more flexibility for input directory structure to LoadIncrementalHFiles"
...
This reverts commit ee3accb370
2018-01-11 10:42:17 -08:00
c88e444408
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout.
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-11 10:34:54 -08:00
15bb49d803
Revert due to missing author. "HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout."
...
This reverts commit 0b9dc14dbc
2018-01-11 10:34:42 -08:00
0b9dc14dbc
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout.
2018-01-11 10:31:45 -08:00
71a1192d67
HBASE-19751 Use RegionInfo directly instead of an identifier and a namespace when getting WAL
2018-01-11 15:47:34 +08:00
8bf6adbe75
HBASE-19749 Revisit logic of UserScanQueryMatcher#mergeFilterResponse method
2018-01-11 12:05:02 +08:00
b568cf4ebd
HBASE-19740 Repeated error message for NamespaceExistException
2018-01-10 18:25:45 -08:00
ee3accb370
Add more flexibility for input directory structure to LoadIncrementalHFiles
2018-01-10 13:37:14 -08:00
67a2c62a6a
HBASE-19748 TestRegionReplicaFailover and TestRegionReplicaReplicationEndpoint UT hangs
2018-01-10 18:17:35 +08:00
09c11b0ca4
HBASE-19743 Disable TestMemstoreLABWithoutPool
...
This is a revert of a revert, i.e., a reapplication only this time
I'm pushing w/ the right JIRA number
2018-01-09 18:21:38 -08:00
88864f3d5d
Revert "HBASE-19742 Disable TestMemstoreLABWithoutPool"
...
Revert because have wrong JIRA number.
This reverts commit 1454c44bcb
2018-01-09 18:21:20 -08:00
669b08f8ec
HBASE-19742 Disable TestMemstoreLABWithoutPool
2018-01-09 14:33:29 -08:00
81ea657ed1
HBASE-19483 Add proper privilege check for rsgroup commands addendum
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-09 07:58:20 -08:00
f458b89c05
HBASE-19139 Create Async Admin methods for Clear Block Cache
2018-01-09 14:43:01 +08:00
eb88b69845
HBASE-19719 Fix checkstyle issues
2018-01-09 13:11:01 +08:00
2d5267331e
HBASE-19711 TestReplicationAdmin.testConcurrentPeerOperations hangs
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
ec364d0d6c
HBASE-19707 Race in start and terminate of a replication source after we async start replicatione endpoint
2018-01-09 13:11:01 +08:00
01f1fc8284
HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
1a89793ee8
HBASE-19634 Add permission check for executeProcedures in AccessController
2018-01-09 13:11:01 +08:00
8eeedf0403
HBASE-19697 Remove TestReplicationAdminUsingProcedure
2018-01-09 13:11:01 +08:00
a5cfc51cbe
HBASE-19661 Replace ReplicationStateZKBase with ZKReplicationStorageBase
2018-01-09 13:11:01 +08:00
368db315a6
HBASE-19687 Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore
2018-01-09 13:11:01 +08:00
d95ee41583
HBASE-19544 Add UTs for testing concurrent modifications on replication peer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
d357920988
HBASE-19686 Use KeyLocker instead of ReentrantLock in PeerProcedureHandlerImpl
2018-01-09 13:11:01 +08:00
a85a215f36
HBASE-19623 Create replication endpoint asynchronously when adding a replication source
2018-01-09 13:11:01 +08:00
19707a85dd
HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer
2018-01-09 13:11:01 +08:00
f89920a60f
HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface
2018-01-09 13:11:01 +08:00
62a4f5bb46
HBASE-19635 Introduce a thread at RS side to call reportProcedureDone
2018-01-09 13:11:01 +08:00
f4703c6ed3
HBASE-19617 Remove ReplicationQueues, use ReplicationQueueStorage directly
2018-01-09 13:11:01 +08:00
5fc90244a8
HBASE-19642 Fix locking for peer modification procedure
2018-01-09 13:11:01 +08:00
fdc907140b
HBASE-19592 Add UTs to test retry on update zk failure
2018-01-09 13:11:01 +08:00
1025388da6
HBASE-19630 Add peer cluster key check when add new replication peer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
65eb91f84e
HBASE-19573 Rewrite ReplicationPeer with the new replication storage interface
2018-01-09 13:11:01 +08:00
c4fa568b47
HBASE-19599 Remove ReplicationQueuesClient, use ReplicationQueueStorage directly
2018-01-09 13:11:01 +08:00
5e6c303528
HBASE-19543 Abstract a replication storage interface to extract the zk specific code
2018-01-09 13:11:01 +08:00
8f5e54a456
HBASE-19525 RS side changes for moving peer modification from zk watcher to procedure
2018-01-09 13:11:01 +08:00
7afae59323
HBASE-19580 Use slf4j instead of commons-logging in new, just-added Peer Procedure classes
2018-01-09 13:11:01 +08:00
17762c60cc
HBASE-19520 Add UTs for the new lock type PEER
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
62496b5f36
HBASE-19564 Procedure id is missing in the response of peer related operations
2018-01-09 13:11:01 +08:00
7f4bd0d371
HBASE-19524 Master side changes for moving peer modification from zk watcher to procedure
2018-01-09 13:11:01 +08:00
f17198ff19
HBASE-19216 Implement a general framework to execute remote procedure on RS
2018-01-09 13:11:01 +08:00
7ddf79946d
HBASE-19483 Add proper privilege check for rsgroup commands
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-08 19:48:37 -08:00
c5277d5f88
HBASE-19729 UserScanQueryMatcher#mergeFilterResponse should return INCLUDE_AND_SEEK_NEXT_ROW when filterResponse is INCLUDE_AND_SEEK_NEXT_ROW
2018-01-09 07:19:40 +08:00
2509a150c0
HBASE-19731 TestFromClientSide#testCheckAndDeleteWithCompareOp and testNullQualifier are flakey
2018-01-08 10:35:36 -08:00
2af61718b1
HBASE-19541 Removed unnecessary semicolons in hbase-server
2018-01-08 16:43:43 +01:00
afc2cdbaff
HBASE-19684 BlockCacheKey toString Performance
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-08 17:06:21 +08:00
7378dad5a9
HBASE-19712 Fix TestSnapshotQuotaObserverChore#testSnapshotSize
2018-01-08 08:47:18 +08:00
5a66eb978c
HBASE-19696 Filter returning INCLUDE_AND_NEXT_COL doesn't skip remaining versions when scan has explicit columns (Ankit Singhal)
2018-01-07 07:50:00 -08:00
45e46bb242
HBASE-19709 Ensure that we don't set a poolSize of 0
...
In some situations, Runtime.getRuntime().getAvailableProcessors()
may return 0 which would result in calculatePoolSize returning 0
which will trigger an exception. Guard against this case.
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-01-05 18:44:35 -05:00
a30d9fe8d9
HBASE-19651 Remove LimitInputStream
2018-01-05 16:59:29 -06:00
2bf2b5b2c1
Revert "Revert "Revert "HBASE-19651 Remove LimitInputStream"""
...
This reverts commit 032701c96c
2018-01-05 16:58:26 -06:00
032701c96c
Revert "Revert "HBASE-19651 Remove LimitInputStream""
...
This reverts commit e5f7030ded
2018-01-05 16:55:59 -06:00
e5f7030ded
Revert "HBASE-19651 Remove LimitInputStream"
...
Revert to put Beluga in place as author.
This reverts commit 032fdc53de
2018-01-05 16:55:01 -06:00
9cd0e1a490
HBASE-17436 Add column to master web UI for displaying region states when listing regions of a table.
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-05 14:32:27 -08:00
5ce9351129
HBASE-19483 Add proper privilege check for rsgroup commands - revert due to pending review comment
2018-01-05 12:06:31 -08:00
fc7736eb00
HBASE-19483 Add proper privilege check for rsgroup commands
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-05 11:18:07 -08:00
032fdc53de
HBASE-19651 Remove LimitInputStream
...
Signed-off-by: Beluga Behr <dam6923@gmail.com>
2018-01-05 08:31:11 -06:00
bff937a767
HBASE-19688 TimeToLiveProcedureWALCleaner should extends BaseLogCleanerDelegate
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-04 10:26:23 -08:00
8119acfca7
HBASE-19596 RegionMetrics/ServerMetrics/ClusterMetrics should apply to all public classes
2018-01-04 13:13:34 +08:00
338a74e737
HBASE-19490 Rare failure in TestRateLimiter
2018-01-04 10:46:43 +08:00
301062566a
HBASE-19613 Miscellaneous changes to WALSplitter.
...
* Use ArrayList instead LinkedList
* Use Apache Commons where appropriate
* Parameterize and improve logging
2018-01-03 18:30:10 -08:00
6e136f26bf
HBASE-18806 VerifyRep by snapshot need not to restore snapshot for each mapper
2018-01-04 10:10:03 +08:00
9a98bb4ce9
HBASE-19691 Removes Global(A) requirement for getClusterStatus
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-03 18:47:53 -05:00
d8ef30c5e1
HBASE-19654 Remove misleading and chatty debug message in ReplicationLogCleaner
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-03 06:58:32 -08:00
ab0e459693
HBASE-19620 Add UT to confirm the race in MasterRpcServices.getProcedureResult
2018-01-03 20:29:07 +08:00
d84c4ec29f
HBASE-19666 TestDefaultCompactSelection test failed
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-03 19:08:50 +08:00
a47afc84cd
HBASE-19675 Miscellaneous HStore Class Improvements.
...
* Use HashSet instead of List for a variable which is only used for lookups
* Remove logging code guards in favor of slf4j parameters
* Use CollectionsUtils.isEmpty() consistently
* Small check-style fixes
2018-01-02 15:16:53 -08:00
32e0107751
HBASE-19667 Get rid of MasterEnvironment#supportGroupCPs
2018-01-03 04:59:38 +08:00
f6f57d38f7
HBASE-19358 Improve the stability of splitting log when do fail over
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-01-03 04:11:40 +08:00
9c2a35542f
HBASE-19676 CleanerChore logging improvements (BELUGA BEHR)
2018-01-02 06:26:29 -08:00
9ac95419dc
HBASE-19658: Fixing small problem and enabling test
2018-01-02 12:51:20 +02:00
6a0e6fefd3
HBASE-19671 Fix TestMultiParallel#testActiveThreadsCount
2018-01-02 16:57:14 +08:00
6708d54478
HBASE-19679 Superusers Logging and Data Structures (BELUGA BEHR)
2018-01-01 14:18:21 -08:00
73ab51e946
HBASE-19678 HBase Admin security capabilities should be represented as a Set - revert due to wrong issue
2018-01-01 14:16:01 -08:00
6c2aa4c9cc
HBASE-19678 HBase Admin security capabilities should be represented as a Set (BELUGA BEHR)
2017-12-31 11:31:44 -08:00
360d465a4a
HBASE-19677 Miscellaneous HFileCleaner Improvements (BELUGA BEHR)
2017-12-31 09:47:12 -08:00
0cd6050d09
HBASE-19672 Correct comments for default values of major compaction in SortedCompactionPolicy#getNextMajorCompactTime()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-30 16:27:20 -08:00
0d0964aa6a
HBASE-19649 Use singleton feature for ImmutableSegment
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-30 06:34:45 +08:00
a21eb68f95
HBASE-8518 Get rid of hbase.hstore.compaction.complete setting
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-30 04:23:18 +08:00
e23f7afe57
HBASE-19428 Deprecate the compareTo(Row)
2017-12-29 20:03:39 +08:00
8d0da1a77f
HBASE-19282: Making CellChunkMap the default index (CellSet delegatee) for ImmutableSegments, when MSLAB is used.
...
In order to avoid additional user settings. If no MSLAB is requested the index is going to be CellArrayMap
Signed-off-by: Anastasia Braginsky <anastas@yahoo-inc.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-28 16:26:38 -08:00
b3a4fca48f
HBASE-19660 Up default retries from 10 to 15 and blocking store files limit from 10 to 16
2017-12-28 14:59:40 -08:00
2dae9d16e2
Revert "HBASE-19651 Remove LimitInputStream"
...
This reverts commit 28eaf715e8
2017-12-28 14:31:44 -08:00
28eaf715e8
HBASE-19651 Remove LimitInputStream
...
Signed-off-by: Beluga Behr <dam6923@gmail.com>
2017-12-28 13:04:41 -08:00
0c4b520d97
HBASE-19133 Transfer big cells or upserted/appended cells into
...
MSLAB upon flattening to CellChunkMap
Signed-off-by: Gali Sheffi <gsheffi@oath.com>
2017-12-28 10:51:44 -08:00
c3b4f788b1
HBASE-19552 find-and-replace thirdparty offset
2017-12-28 11:52:32 -06:00
ea7d5fc884
HBASE-19552 Update hbase-thirdparty version
...
Some manual cleanup of changing package names in pom files and getting
rid of the no-longer-needed netty system property.
This commit will break compilation, package renames in source code are
done in follow-on commits using straightforward find and replace.
's/org.apache.hadoop.hbase.shaded.com.google/org.apache.hbase.thirdparty.com.google/'
's/org.apache.hadoop.hbase.shaded.io.netty/org.apache.hbase.thirdparty.io.netty/'
2017-12-28 11:41:42 -06:00
8580ba7b6f
HBASE-19643 Need to update cache location when get error in AsyncBatchRpcRetryingCaller
2017-12-28 14:33:24 +08:00
53233be5a9
HBASE-19626 Rename Cell.DataType to Cell.Type
2017-12-28 13:04:29 +08:00
1050936ce7
HBASE-19652 Turn down CleanerChore logging; too chatty
2017-12-27 20:49:26 -08:00
2c65f039ed
HBASE-19647 Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level; ADDENDUM
2017-12-27 14:14:58 -08:00
01b1f48ccd
HBASE-19644 add the checkstyle rule to reject the illegal imports
2017-12-28 04:10:42 +08:00
16cd916976
HBASE-19647 Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level
2017-12-27 11:55:42 -08:00
6b39062e86
HBASE-19628 ByteBufferCell should extend ExtendedCell
2017-12-27 22:26:40 +08:00
467a4667d8
HBASE-19629 RawCell#getTags should return the Iterator<Tag> in order to avoid iterating through whole tag array at once
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 21:33:42 +08:00
0e85a880fb
HBASE-19615 CompositeImmutableSegment ArrayList Instead of LinkedList
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 10:04:53 +08:00
0e7fff5829
HBASE-19624 TestIOFencing hangs
2017-12-27 09:33:53 +08:00
7ce1943ef3
HBASE-19550 Wrap the cell passed via Mutation#add(Cell) to be of ExtendedCell
2017-12-26 16:39:51 +08:00
1556939236
HBASE-19621 Revisit the methods in ReplicationPeerConfigBuilder
2017-12-26 14:40:31 +08:00
2ce5dc8927
HBASE-19618 Remove replicationQueuesClient.class/replicationQueues.class config and remove table based ReplicationQueuesClient/ReplicationQueues implementation
2017-12-26 14:39:41 +08:00
27c56860da
HBASE-19608 Race in MasterRpcServices.getProcedureResult
2017-12-25 11:52:34 +08:00
2f25589422
HBASE-19602 Cleanup the usage of ReplicationAdmin from document
2017-12-23 18:55:54 +08:00
0dadb45ad7
HBASE-19576 Introduce builder for ReplicationPeerConfig and make it immutable
2017-12-23 18:55:29 +08:00
c0c802f62b
HBASE-19514 TestJMXListener now uses random port
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2017-12-22 18:40:58 +01:00
c0598dcb10
HBASE-19593 Possible NPE if wal is closed during waledit append.(Rajeshabbu)
2017-12-22 17:20:45 +05:30
448ba3a78f
HBASE-19496 Reusing the ByteBuffer in rpc layer corrupt the ServerLoad and RegionLoad
2017-12-22 18:54:25 +08:00
001bce54e7
HBASE-19590 Remove the duplicate code in deprecated ReplicationAdmin
2017-12-22 18:32:15 +08:00
8ec0aa0d70
HBASE-19589 New regions should always be added with state CLOSED (followup of HBASE-19530)
2017-12-21 22:57:53 -08:00
37bf54a516
HBASE--17248 fix javadoc in SimpleRegionNormalizer
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-21 21:35:08 -08:00
bcaf2fd20c
HBASE-19148 Reevaluate default values of configurations; minor ADDENDUM
2017-12-21 14:14:07 -08:00
4d6b928682
HBASE-19148 Reevaluate default values of configurations
...
Removed unused:
<name>hbase.fs.tmp.dir</name>
Added hbase.master.loadbalance.bytable
Edit of description text. Moved stuff around to put configs beside each
other.
M hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerCommandLine.java
Emit some hbase configs in log on startup.
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-21 14:07:16 -08:00
59baf12c03
HBASE-19575 add copy constructor to Mutation
2017-12-22 03:24:43 +08:00
35728acd21
HBASE-19578 MasterProcWALs cleaning is incorrect
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-21 09:38:25 -08:00
00b0288c96
HBASE-10092 Addendum fix TestAssignProcedure and TestWALEntrySinkFilter
2017-12-21 21:43:17 +08:00
9a07e07e55
HBASE-19559 Fix TestLogRolling.testLogRollOnDatanodeDeath
2017-12-21 15:20:04 +08:00
f572c4b80e
HBASE-10092 Move up on to log4j2
...
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test
This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:21:33 -08:00
7f938dd980
HBASE-19218 Master stuck thinking hbase:namespace is assigned after restart preventing intialization
...
Signed-off-by: Li Xiang <easyliangjob@gmail.com>
2017-12-20 21:47:10 -08:00
c811d7f965
HBASE-19542 fix TestSafemodeBringsDownMaster
2017-12-21 10:32:14 +08:00
4d71f4e32f
HBASE-19567 ClassNotFoundException:
...
org.apache.hadoop.hbase.KeyValue$RawBytesComparator starting 2.0.0 over
a 0.98.25 data.
2017-12-20 11:21:06 -08:00
084e324fb9
HBASE-19554 For debug: Modify HTU.waitUntilAllRegionsAssigned to handle the case where we do not have entries for the given table
2017-12-20 10:15:18 -08:00
55fefd4b5a
HBASE-19561 maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When dealing with each region
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-20 07:05:05 -08:00
cae5c2706b
HBASE-19492 (addendum) Add EXCLUDE_NAMESPACE and EXCLUDE_TABLECFS support to replication peer config
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-12-20 20:49:05 +08:00
d2534fc570
HBASE-19468 FNFE during scans and flushes (Ram)
2017-12-20 11:09:12 +05:30
3c78ad5215
HBASE-19558 TestRegionsOnMasterOptions hack so it works reliably
2017-12-19 15:59:02 -08:00
70608acf28
HBASE-19556: Remove TestAssignmentManager#testGoodSplit, which no longer make sense
2017-12-19 13:55:05 -08:00
32f6fd41c2
HBASE-19494 Create simple WALKey filter that can be plugged in on the Replication Sink
...
Implement new WALEntrySinkFilter (as opposed to WALEntryFilter) and
specify the implmentation (with a no-param constructor) in config
using property hbase.replication.sink.walentrysinkfilter
Signed-off-by: wolfgang hoschek whoscheck@cloudera.com
2017-12-19 13:48:59 -08:00
dbe409e2c7
HBASE-19555 TestSplitTransactionOnCluster is flaky
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-19 09:25:57 -08:00
03e79b7994
HBASE-19492 Add EXCLUDE_NAMESPACE and EXCLUDE_TABLECFS support to replication peer config
2017-12-19 16:53:43 +08:00
7a7e55b601
HBASE-19532 AssignProcedure#COMPARATOR may produce incorrect sort order
2017-12-18 18:32:24 -08:00
b4056d267a
HBASE-19122 Suspect methods on Cell to be deprecated
2017-12-18 15:20:44 -08:00
9d0c7c6dfb
HBASE-19531 Remove needless volatile declaration
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-19 00:34:28 +08:00
c8bf03f5fb
HBASE-19521 HBase mob compaction needs to check hfile version
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-18 08:32:18 -08:00
62b591bdc7
HBASE-19546 TestMasterReplication.testCyclicReplication2 uses wrong assertion
2017-12-18 19:59:14 +08:00
988ea870ed
HBASE-18309 (addendum) fix NPE in LogCleaner
2017-12-17 17:33:09 +08:00
979767824d
HBASE-19522 The complete order may be wrong in AsyncBufferedMutatorImpl
2017-12-16 14:00:59 +08:00
59529a78f0
HBASE-19498 Fix findbugs and error-prone warnings in hbase-client (branch-2)
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-12-15 15:55:51 -08:00
20b42d2d70
HBASE-18352 Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
...
Reenables the test. Adds facility to HBaseTestingUtility so
you can pass in ports a restarted cluster should use. This
is needed so retention of region placement, on which this
test depends, can come trigger (this is why it was broke
on AMv2 commit... region placement retention is done
different in AMv2).
2017-12-15 12:47:43 -08:00
89e2869e22
HBASE-19272 Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK
...
works again...
I removed them
2017-12-15 12:19:49 -08:00
75f512bd71
HBASE-18838 Fix hadoop3 check-shaded-invariants
2017-12-15 11:19:47 -06:00
010012cbcb
HBASE-18946 Stochastic load balancer assigns replica regions to the same RS
...
Added new bulk assign createRoundRobinAssignProcedure to complement
the existing createAssignProcedure. The former asks the balancer for
target servers to set into the created AssignProcedures. The latter
sets no target server into AssignProcedure. When no target server
is specified, we make effort at assign-time at trying to deploy the
region to its old location if there was one.
The new round robin assign procedure creator does not do this. Use
the new round robin method on table create or reenabling offline
regions. Use the old assign in ServerCrashProcedure or in
EnableTable so there is a chance we retain locality.
Bulk preassigning passing all to-be-assigned to the balancer in one
go is good for ensuring good distribution especially when read
replicas in the mix.
The old assign was single-assign scoped so region replicas could
end up on the same server.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
Cleanup around forceNewPlan. Was confusing.
Added a Comparator to sort AssignProcedures so meta and system tables
come ahead of user-space tables.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Remove the forceNewPlan argument on createAssignProcedure. Didn't make
sense given we were creating a new AssignProcedure; the arg had no
effect.
(createRoundRobinAssignProcedures) Recast to feed all regions to the balancer in
bulk and to sort the return so meta and system tables take precedence.
Miscellaneous fixes including keeping the Master around until all
RegionServers are down, documentation on how assignment retention
works, etc.
2017-12-15 08:53:41 -08:00
11e82de8a1
HBASE-15536 Make AsyncFSWAL as our default WAL
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-12-15 20:45:29 +08:00
018a65b947
HBASE-19505 Disable ByteBufferPool by default at HM.
2017-12-15 15:43:10 +05:30
deba43b156
HBASE-19516 IntegrationTestBulkLoad and IntegrationTestImportTsv run into 'java.lang.RuntimeException: DistributedHBaseCluster@1bb564e2 not an instance of MiniHBaseCluster' (Ankit)
2017-12-14 18:37:09 -08:00
661491b56b
HBASE-19513 Fix the wrapped AsyncFSOutput implementation
2017-12-15 09:39:36 +08:00
6ab8ce9829
Revert "HBASE-18352 Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas"
...
Pushed prematurely
This reverts commit 1a173f820b
2017-12-14 14:17:20 -08:00
70f02dbc7c
HBASE-19462 Deprecate all addImmutable methods in Put
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-14 14:15:54 -08:00
1a173f820b
HBASE-18352 Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas
...
disabled by Proc-V2 AM in HBASE-14614
Reenables the test. Adds facility to HBaseTestingUtility so
you can pass in ports a restarted cluster should use. This
is needed so retention of region placement, on which this
test depends, can come trigger (this is why it was broke
on AMv2 commit... region placement retention is done
different in AMv2).
2017-12-14 14:13:55 -08:00
Chia-Ping Tsai
Michael Stack
BELUGA BEHR
gsheffi
Sean Busbey
zhangduo
Vasudevan
tedyu
Toshihiro Suzuki
Apekshit Sharma
Umesh Agashe
Josh Elser
Esteban Gutierrez
Andrew Purtell
Mike Drob
anastas
Reid Chan
eshcar
Thiruvel Thirumoolan
Pankaj Kumar
huzheng
Guanghao Zhang
Aman Poonia
Balazs Meszaros
Rajeshbabu Chintaguntla
Ashish Singhi
Peter Somogyi
kewang
Rahul Gidwani
binlijin
Jan Hentschel
haxiaolin
Sergey Soldatov
Artem Ervits
Guangxu Cheng
lixiang
Scott Wilson
Rahul Gidwani
Janos Gub
Jingyun Tian
Xiang Li
brandboat
Daisuke
Yi Liang
Yun Chi-Shih
QilinCao
anoopsamjohn